now.data_loading.create_dataclass module

now.data_loading.create_dataclass.update_dict_with_no_overwrite(dict1, dict2)

Update dict1 with dict2, but only if the key does not exist in dict1.

Parameters

• dict1 (Dict) – dict to be updated

• dict2 (Dict) – dict to be used for updating

now.data_loading.create_dataclass.create_dataclass(fields=None, fields_modalities=None, dataset_type=None, user_input=None)

Create a dataclass from the selected index fields and their corresponding modalities or directly from the user input which should contain that information. If both are provided, the user input will be used.

for example: the index fields modalities can be: {‘test.txt’: Text , ‘image.png’: Image}

the dataclass will be:

@dataclass class DataClass:

text_0: Text image_0: Image price: float description: str

Parameters

• fields (Optional[List]) – list of fields

• fields_modalities (Optional[Dict]) – dict of fields and their modalities

• dataset_type (Optional[DatasetTypes]) – dataset type

• user_input (Optional[UserInput]) – user inputs

Returns

dataclass object

now.data_loading.create_dataclass.create_annotations_and_class_attributes(fields, fields_modalities, field_names_to_dataclass_fields, dataset_type=None)

Create annotations and class attributes for the dataclass In case of S3 bucket, new field is created to prevent uri loading

Parameters

• fields (List) – list of fields

• fields_modalities (Dict[str, TypeVar]) – dict of fields and their modalities

• field_names_to_dataclass_fields (Dict) – dict of selected field names and their corresponding fields in dataclass

• dataset_type (Optional[DatasetTypes]) – dataset type

now.data_loading.create_dataclass.create_s3_type(modality)

Create a new type for S3 bucket which sets the right modality

now.data_loading.create_dataclass.create_local_text_type()

Create a new type for local text which sets the right modality and loads from URI

now.data_loading.create_dataclass.create_blob_type(modality)

Creates a new type which loads into blob instead of tensor

now.data_loading.create_dataclass.create_dataclass_fields_file_mappings(fields, fields_modalities)

Create a mapping between the dataclass fields and the file fields

Parameters

• fields (List) – list of fields

• fields_modalities (Dict) – dict of fields and their modalities