now.data_loading.es.connector module#

class now.data_loading.es.connector.ElasticsearchConnector(connection_str='http://localhost:9200', connection_args=None)[source]#

Bases: object

Provides an interface to an Elasticsearch database. :type connection_str: str :param connection_str: A connection string for the ES instance. Usually, it

includes url, port, username, password, etc. Typically, it has the form: ‘https://{user_name}:{password}@{host}:{port}’

Parameters

connection_args (Optional[Dict]) – Dictionary with additional connection arguments, e.g., information about certificates

get_documents(index_name, page_size=10)[source]#

Returns all documents stored in a specific index :type index_name: str :param index_name: Name of the index in the Elasticsearch database :type page_size: Optional[int] :param page_size: To retrieve a large number of documents, multiple request are

executed, where each request returns a spefic number of documents (max. 10,000). The page_size refers to this number.

Return type

List[Dict]

Returns

Documents in the form of a list of dictionaries

get_documents_by_query(query, index_name, page_size=10)[source]#

Executes an Elasticsearch query on a given index and returns a generator which yields pages of documents from the query results. :type query: Dict :param query: Elasticsearch query :type index_name: str :param index_name: Name of an Elasticsearch index :type page_size: Optional[int] :param page_size: Number of documents per page :rtype: Generator[List[Dict], None, None] :return: Generator which yields one page of documents on each call.

close()[source]#

Closes Elasticsearch connection.

Return type

None