Public Member Functions | Protected Member Functions | Private Member Functions | Private Attributes | List of all members
Aws::FileManagement::FileManagerStrategy Class Reference

#include <file_manager_strategy.h>

Inheritance diagram for Aws::FileManagement::FileManagerStrategy:
Inheritance graph

Public Member Functions

 FileManagerStrategy (const FileManagerStrategyOptions &options)
bool isDataAvailable () override
bool isDeleteStaleData () override
DataToken read (std::string &data) override
void resolve (const DataToken &token, bool is_success) override
bool shutdown () override
bool start () override
void write (const std::string &data) override
 ~FileManagerStrategy () override=default
- Public Member Functions inherited from Aws::FileManagement::DataManagerStrategy
 DataManagerStrategy ()=default
 ~DataManagerStrategy () override=default
- Public Member Functions inherited from Service
ServiceState getState ()
virtual std::string getStatusString ()
 Service ()
virtual ~Service ()=default

Protected Member Functions

std::string getActiveWriteFile ()
std::string getFileToRead ()
- Protected Member Functions inherited from Service
void setState (ServiceState new_state)

Private Member Functions

void addFilePathToStorage (const std::experimental::filesystem::path &file_path)
void checkIfStorageLimitHasBeenReached (const uintmax_t &new_data_size)
void checkIfWriteFileShouldRotate (const uintmax_t &new_data_size)
void deleteFile (const std::string &file_path)
void deleteOldestFile ()
void discoverStoredFiles ()
void initializeStorage ()
void initializeTokenStore ()
void rotateWriteFile ()
void validateOptions ()

Private Attributes

std::string active_read_file_
std::mutex active_read_file_mutex_
std::unique_ptr< std::ifstream > active_read_file_stream_
std::string active_write_file_
std::mutex active_write_file_mutex_
std::atomic< size_t > active_write_file_size_ {}
FileManagerStrategyOptions options_
std::list< std::string > stored_files_
std::atomic< size_t > stored_files_size_ {}
std::unique_ptr< TokenStoretoken_store_

Detailed Description

Manages how files are split up, which files to write to and read when requested.

Definition at line 285 of file file_manager_strategy.h.

Constructor & Destructor Documentation

Aws::FileManagement::FileManagerStrategy::FileManagerStrategy ( const FileManagerStrategyOptions options)

Definition at line 231 of file file_manager_strategy.cpp.

Aws::FileManagement::FileManagerStrategy::~FileManagerStrategy ( )

Member Function Documentation

void Aws::FileManagement::FileManagerStrategy::addFilePathToStorage ( const std::experimental::filesystem::path &  file_path)

Adds a file path to the stored_files_ list

file_paththe path to the file

Definition at line 410 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::checkIfStorageLimitHasBeenReached ( const uintmax_t &  new_data_size)

Checks if all the space taken up by offline storage exceeds the setting storage_limit_in_kb. If it has reached that amount it deletes the oldest file from disk.

new_data_sizethe size of all files on disk in bytes

Definition at line 452 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::checkIfWriteFileShouldRotate ( const uintmax_t &  new_data_size)

Checks if the active_write_file_ size is equal to or greater than maximum_file_size_in_kb in size. If it is then it calls rotateWriteFile.

new_data_sizethe size of the active_write_file in bytes

Definition at line 442 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::deleteFile ( const std::string &  file_path)

Delete a file at this path. Used when all data from a file has been uploaded successfully or when the offline file storage has reached storage_limit_in_kb

file_pathpath to the file to be deleted

Definition at line 383 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::deleteOldestFile ( )

Deletes the oldest file in offline storage. Cannot delete the active_write_file_

Definition at line 461 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::discoverStoredFiles ( )

Finds any offline storage files on disk that are awaiting upload.

Definition at line 370 of file file_manager_strategy.cpp.

std::string Aws::FileManagement::FileManagerStrategy::getActiveWriteFile ( )

Returns the active write file. Getter used for testing.

the file the system is actively writing to

Definition at line 359 of file file_manager_strategy.h.

std::string Aws::FileManagement::FileManagerStrategy::getFileToRead ( )

Returns the next file from storage that should be read from. Prioritizes newest files first.

a path to the file to be read
std::runtime_errorIf there are no files available to read

Definition at line 390 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::initializeStorage ( )

Initializes the folder that offline data will be saved to, creating it if it doesn't already exist.

Definition at line 250 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::initializeTokenStore ( )

Creates the TokenStore and runs its initialize method

Definition at line 260 of file file_manager_strategy.cpp.

bool Aws::FileManagement::FileManagerStrategy::isDataAvailable ( )

Returns true if there is offline data on disk awaiting to be uploaded, false otherwise.

bool if there is data available

Implements Aws::FileManagement::DataManagerStrategy.

Definition at line 267 of file file_manager_strategy.cpp.

bool Aws::FileManagement::FileManagerStrategy::isDeleteStaleData ( )

Deletes stale data that won't be accepted by the CloudWatch API. This is done in a separate method so it doesn't block the current publishing task after reading the data.

Implements Aws::FileManagement::DataManagerStrategy.

Definition at line 276 of file file_manager_strategy.cpp.

DataToken Aws::FileManagement::FileManagerStrategy::read ( std::string &  data)

Reads a line of data from file storage. The most recent data in storage is read first. This returns a DataToken which should be passed to resolve() when the caller is done with it.

std::stringa string of data which is read from storage
DataToken that should be passed to resolve() when done

Implements Aws::FileManagement::DataManagerStrategy.

Definition at line 300 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::resolve ( const DataToken token,
bool  is_success 

Resolves a DataToken. If the data associated with the token was successfully sent then the FileManagerStrategy knows it now longer needs to hold onto it. If the data was not successfully sent then the FileManagerStrategy knows it needs to hold onto it and re-send that data in the next read.

DataTokenthe token to be resolved
is_successwhether the data was successfully sent or not

Implements Aws::FileManagement::DataManagerStrategy.

Definition at line 335 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::rotateWriteFile ( )

Moves the active_write_file into stored_files list and creates a new file setting the active_write_file_ to that new file.

Definition at line 415 of file file_manager_strategy.cpp.

bool Aws::FileManagement::FileManagerStrategy::shutdown ( )

Saves all in flight tokens and information out to disk so that on startup the FileManagerStrategy can start reading from the same file location that it was up to previously.

bool whether it shutdown successfully

Reimplemented from Service.

Definition at line 363 of file file_manager_strategy.cpp.

bool Aws::FileManagement::FileManagerStrategy::start ( )

Starts the FileManagerStrategy, this does any initialization I/O tasks required. It is separate from the constructor to keep the construction of this class lightweight. It checks if there are any existing offline storage files on disk, initializes the token store, and ensures there is a new file ready to write to if uploads fail.

bool whether it started successfully or not

Reimplemented from Service.

Definition at line 238 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::validateOptions ( )

Validates options that were passed to the constructor of this class. Also sanitizes any paths or slightly incorrect options.

Definition at line 246 of file file_manager_strategy.cpp.

void Aws::FileManagement::FileManagerStrategy::write ( const std::string &  data)

Write a line of data to to file storage.

std::stringa string of data to write to storage

Implements Aws::FileManagement::DataManagerStrategy.

Definition at line 280 of file file_manager_strategy.cpp.

Member Data Documentation

std::string Aws::FileManagement::FileManagerStrategy::active_read_file_

The file we are currently reading from. The FileManagerStrategy only reads from one file at a time.

Definition at line 455 of file file_manager_strategy.h.

std::mutex Aws::FileManagement::FileManagerStrategy::active_read_file_mutex_

A lock on the active read file, to ensure that it does not get deleted while it is being read from.

Definition at line 467 of file file_manager_strategy.h.

std::unique_ptr<std::ifstream> Aws::FileManagement::FileManagerStrategy::active_read_file_stream_

An IOStream for the active_read_file. Kept open so that it's easy to move to the next point in the file with each read instead of continually opening a new stream each time.

Definition at line 461 of file file_manager_strategy.h.

std::string Aws::FileManagement::FileManagerStrategy::active_write_file_

Path to the file we're currently writing to. This file cannot be read from or deleted.

Definition at line 438 of file file_manager_strategy.h.

std::mutex Aws::FileManagement::FileManagerStrategy::active_write_file_mutex_

A lock on the active write file, to ensure that when it's rotated nothing is writing to it at the same time.

Definition at line 449 of file file_manager_strategy.h.

std::atomic<size_t> Aws::FileManagement::FileManagerStrategy::active_write_file_size_ {}

The size of the active_write_file. Stored here to minimize disk reads.

Definition at line 443 of file file_manager_strategy.h.

FileManagerStrategyOptions Aws::FileManagement::FileManagerStrategy::options_

Options for how and where to store files, and maximum file sizes.

Definition at line 472 of file file_manager_strategy.h.

std::list<std::string> Aws::FileManagement::FileManagerStrategy::stored_files_

Stored files to read from in order from most recent to oldest.

Definition at line 428 of file file_manager_strategy.h.

std::atomic<size_t> Aws::FileManagement::FileManagerStrategy::stored_files_size_ {}

Disk space used by all stored files. Does not include active_write_file_size_.

Definition at line 433 of file file_manager_strategy.h.

std::unique_ptr<TokenStore> Aws::FileManagement::FileManagerStrategy::token_store_

Stores which tokens to read from.

Definition at line 477 of file file_manager_strategy.h.

The documentation for this class was generated from the following files:

Author(s): AWS RoboMaker
autogenerated on Fri May 7 2021 02:18:23