aiflows.flow_verse package

Submodules

aiflows.flow_verse.loading module

class aiflows.flow_verse.loading.FlowModuleSpec(repo_id: str, revision: str, commit_hash: str, cache_dir: str, sync_dir: str)

Bases: object

This class contains the flow module specification.

Parameters:

  • repo_id (str) – The repository ID

  • revision (str) – The revision

  • commit_hash (str) – The commit hash

  • cache_dir (str) – The cache directory

  • sync_dir (str) – The sync directory

static build_mod_id(repo_id: str, revision: str)

Static method that builds a module ID from a repository ID and a revision.

cache_dir: str
commit_hash: str
property mod_id

Returns the module ID.

repo_id: str
revision: str
sync_dir: str
class aiflows.flow_verse.loading.FlowModuleSpecSummary(sync_root: str, cache_root: str, mods: List[FlowModuleSpec] = None)

Bases: object

This class contains the flow module specification summary.

Parameters:

  • sync_root (str) – The sync root

  • cache_root (str) – The cache root

  • mods (List[FlowModuleSpec], optional) – The modules

add_mod(flow_mod_spec: FlowModuleSpec)

Adds a FlowModuleSpec object to the FlowModuleSpecSummary object.

Parameters:

flow_mod_spec (FlowModuleSpec) – The FlowModuleSpec object to be added.

property cache_root: str

Returns the remote cache root.

Returns:

The remote cache root.

Return type:

str

static from_flow_mod_file(file_path: str) → FlowModuleSpecSummary | None

Reads a flow module file and returns a FlowModuleSpecSummary object.

Parameters:

file_path (str) – The path to the flow module file.

Returns:

A FlowModuleSpecSummary object if the file exists, otherwise None.

Return type:

Optional[“FlowModuleSpecSummary”]

Raises:

ValueError – If the flow module file is invalid.

get_mod(repo_id: str) → FlowModuleSpec | None

Returns the FlowModuleSpec object for the specified repository ID.

Parameters:

repo_id (str) – The repository ID.

Returns:

The FlowModuleSpec object for the specified repository ID, or None if not found.

Return type:

Optional[FlowModuleSpec]

get_mods() → List[FlowModuleSpec]

Returns a list of FlowModuleSpec objects.

Returns:

A list of FlowModuleSpec objects.

Return type:

List[FlowModuleSpec]

serialize() → str

Serializes the FlowModuleSpecSummary object.

Returns:

The serialized FlowModuleSpecSummary object.

Return type:

str

property sync_root: str

Returns the sync root.

Returns:

The sync root.

Return type:

str

aiflows.flow_verse.loading.add_to_sys_path(path)

Adds a path to sys.path if it’s not already there.

Parameters:

path (str) – The path to add

aiflows.flow_verse.loading.create_empty_flow_mod_file(sync_root: str, cache_root: str, overwrite: bool = False) → str

Creates an empty flow module file.

Parameters:

  • sync_root (str) – The sync root

  • cache_root (str) – The cache root

  • overwrite (bool) – Whether to overwrite the existing flow module file. Defaults to False.

Returns:

The path to the flow module file.

Return type:

str

aiflows.flow_verse.loading.create_init_py(base_dir: str)

Creates an __init__.py file in the given directory.

Parameters:

base_dir (str) – The directory to create the __init__.py file in

aiflows.flow_verse.loading.extract_commit_hash_from_cache_mod_dir(cache_mod_dir: str) → str

Extracts the commit hash from a cache directory.

Parameters:

cache_mod_dir (str) – The cache directory

Returns:

The commit hash

Return type:

str

aiflows.flow_verse.loading.fetch_local(repo_id: str, file_path: str, sync_dir: str) → FlowModuleSpec

Fetches a local dependency.

Parameters:

  • repo_id (str) – The repository ID

  • file_path (str) – The file path

  • sync_dir (str) – The sync directory

Returns:

The flow module specification

Return type:

FlowModuleSpec

aiflows.flow_verse.loading.fetch_remote(repo_id: str, revision: str, sync_dir: str, cache_root: str) → FlowModuleSpec

Fetches a remote dependency.

Parameters:

  • repo_id (str) – The repository ID

  • revision (str) – The revision

  • sync_dir (str) – The sync directory

  • cache_root (str) – The cache root

Returns:

The flow module specification

Return type:

FlowModuleSpec

aiflows.flow_verse.loading.is_local_revision(legal_revision: str) → bool

Check if a given revision is a local revision.

Parameters:

legal_revision (str) – A string representing the revision to check.

Returns:

True if the revision is a local revision, False otherwise.

Return type:

bool

aiflows.flow_verse.loading.is_local_sync_dir_valid(sync_dir: str)

Returns True if the sync_dir is a valid local sync dir, False otherwise.

Parameters:

sync_dir (str) – The sync directory

aiflows.flow_verse.loading.is_sync_dir_modified(sync_dir: str, cache_dir: str) → bool

Returns True if the sync_dir is modified compared to the cache_dir, False otherwise.

Parameters:

sync_dir (str) – The sync directory

Cache_dir:

The cache directory

Returns:

True if the sync_dir is modified compared to the cache_dir, False otherwise

Return type:

bool

Removes a directory or a link.

Parameters:

sync_dir – The directory or link to remove

aiflows.flow_verse.loading.retrive_commit_hash_from_remote(repo_id: str, revision: str) → str

Retrieves the commit hash from a remote repository.

Parameters:

  • repo_id (str) – The repository ID

  • revision (str) – The revision

Returns:

The commit hash

Return type:

str

aiflows.flow_verse.loading.sync_dependencies(dependencies: List[Dict[str, str]], all_overwrite: bool = False) → List[str]

Synchronizes dependencies. (uses the _sync_dependencies function)

Parameters:

  • dependencies (List[Dict[str, str]]) – The dependencies to synchronize

  • all_overwrite (bool) – Whether to overwrite all existing modules or not

Returns:

A list of sync directories

Return type:

List[str]

aiflows.flow_verse.loading.sync_local_dep(previous_synced_flow_mod_spec: FlowModuleSpec | None, repo_id: str, mod_name: str, revision: str, caller_module_name: str, sync_root: str, overwrite: bool = False) → FlowModuleSpec

Synchronize a local dependency.

Parameters:

  • previous_synced_flow_mod_spec (Optional[FlowModuleSpec]) – The previously synced flow module specification.

  • repo_id (str) – The ID of the repository.

  • mod_name (str) – The name of the module.

  • revision (str) – The revision of the module.

  • caller_module_name (str) – The name of the caller module.

  • overwrite (bool) – Whether to overwrite the previously synced flow module specification. Defaults to False.

Returns:

The synced flow module specification.

Return type:

FlowModuleSpec

aiflows.flow_verse.loading.sync_remote_dep(previous_synced_flow_mod_spec: FlowModuleSpec | None, repo_id: str, mod_name: str, revision: str, caller_module_name: str, sync_root: str, cache_root: str = '/Users/nicolasbaldwin/.cache/aiflows/flow_verse', overwrite: bool = False) → FlowModuleSpec

Synchronizes a remote dependency.

Parameters:

  • previous_synced_flow_mod_spec (Optional[FlowModuleSpec]) – The previously synced flow module specification.

  • repo_id (str) – The ID of the repository.

  • mod_name (str) – The name of the module.

  • revision (str) – The revision of the module.

  • caller_module_name (str) – The name of the caller module.

  • cache_root (str) – The root directory of the cache. Defaults to DEFAULT_CACHE_PATH.

  • overwrite (bool) – Whether to overwrite the existing module or not. Defaults to False.

Returns:

The synced flow module specification.

Return type:

FlowModuleSpec

aiflows.flow_verse.loading.validate_and_augment_dependency(dependency: Dict[str, str], caller_module_name: str) → bool

Validates and augments a dependency dictionary.

Parameters:

  • dependency (Dict[str, str]) – A dictionary containing information about the dependency.

  • caller_module_name (str) – The name of the calling module.

Returns:

True if the dependency is local, False otherwise.

Return type:

bool

aiflows.flow_verse.loading.write_flow_mod_summary(flow_mod_summary_path: str, flow_mod_summary: FlowModuleSpecSummary)

Writes a flow module summary to a file.

Parameters:

  • flow_mod_summary_path (str) – The path to the flow module summary file.

  • flow_mod_summary (FlowModuleSpecSummary) – The flow module summary.

aiflows.flow_verse.loading.write_or_append_gitignore(sync_dir: str, mode: str, content: str)

Writes or appends a .gitignore file to the given directory.

Parameters:

  • sync_dir (str) – The directory to write the .gitignore file to

  • mode (str) – The mode to open the file with

  • content (str) – The content to write to the file

aiflows.flow_verse.utils module

aiflows.flow_verse.utils.build_hf_cache_path(repo_id: str, commit_hash: str, cache_root: str) → str

Builds the path to the cache directory for a given Hugging Face model. The path is constructed as follows: {CACHE_ROOT}/models–{username}–{modelname}/snapshots/{commit_hash}

Parameters:

  • repo_id (str) – The repository ID in the format of “username/modelname”.

  • commit_hash (str) – The commit hash of the model snapshot.

  • cache_root (str) – The root directory of the cache.

Returns:

The path to the cache directory for the given model snapshot.

Return type:

str

aiflows.flow_verse.utils.is_local_revision(revision: str)

Returns True if the revision is a local revision, False otherwise.

Parameters:

revision (str) – The revision to check

Returns:

True if the revision is a local revision, False otherwise

Return type:

bool

Module contents