now.data_loading.create_dataclass module#

now.data_loading.create_dataclass.update_dict_with_no_overwrite(dict1, dict2)[source]#

Update dict1 with dict2, but only if the key does not exist in dict1

Parameters
  • dict1 (Dict) – dict to be updated

  • dict2 (Dict) – dict to be used for updating

now.data_loading.create_dataclass.create_dataclass(fields=None, fields_modalities=None, dataset_type=None, user_input=None)[source]#

Create a dataclass from the selected index and filter fields and their corresponding modalities or directly from the user input which should contain that information. If both are provided, the user input will be used.

for example: the index fields modalities can be: {‘test.txt’: Text , ‘image.png’: Image} the filter fields modalities can be: {‘price’: float, ‘description’: str}

the dataclass will be:

@dataclass class DataClass:

text_0: Text image_0: Image price: float description: str

Parameters
  • fields (Optional[List]) – list of fields

  • fields_modalities (Optional[Dict]) – dict of fields and their modalities

  • dataset_type (Optional[DatasetTypes]) – dataset type

  • user_input (Optional[UserInput]) – user inputs

Returns

dataclass object

now.data_loading.create_dataclass.create_annotations_and_class_attributes(fields, fields_modalities, field_names_to_dataclass_fields, dataset_type=None)[source]#

Create annotations and class attributes for the dataclass In case of S3 bucket, new field is created to prevent uri loading

Parameters
  • fields (List) – list of fields

  • fields_modalities (Dict) – dict of fields and their modalities

  • field_names_to_dataclass_fields (Dict) – dict of selected field names and their corresponding fields in dataclass

  • dataset_type (Optional[DatasetTypes]) – dataset type

now.data_loading.create_dataclass.create_s3_type()[source]#

Create a new type for S3 bucket

now.data_loading.create_dataclass.create_dataclass_fields_file_mappings(fields, fields_modalities)[source]#

Create a mapping between the dataclass fields and the file fields

Parameters
  • fields (List) – list of fields

  • fields_modalities (Dict) – dict of fields and their modalities