now.executor.indexer.elastic.es_converter module#

now.executor.indexer.elastic.es_converter.convert_es_to_da(result, get_score_breakdown)[source]#

Transform Elasticsearch documents into DocumentArray. Assumes that all Elasticsearch documents have a ‘text’ field. It returns embeddings as part of the tags for each field that is encoded.

  • result (Union[Dict, List[Dict]]) – results from an Elasticsearch query.

  • get_score_breakdown (bool) – whether to return the embeddings as tags for each document.

Return type



a DocumentArray containing all results.

now.executor.indexer.elastic.es_converter.convert_doc_map_to_es(docs_map, index_name, encoder_to_fields)[source]#

Transform a dictionary (mapping encoder to DocumentArray) into a list of Elasticsearch documents. The docs_map dictionary is expected to have the following structure: {

‘encoder1’: DocumentArray([…]), ‘encoder2’: DocumentArray([…]), # same number of documents as encoder1 …


  • docs_map (Dict[str, DocumentArray]) – dictionary mapping encoder to DocumentArray.

  • index_name (str) – name of the index to be used in Elasticsearch.

  • encoder_to_fields (dict) – dictionary mapping encoder to fields.

Return type



a list of Elasticsearch documents as dictionaries ready to be indexed.

now.executor.indexer.elastic.es_converter.get_base_es_doc(doc, index_name)[source]#
Return type


now.executor.indexer.elastic.es_converter.convert_es_results_to_matches(query_doc, es_results, get_score_breakdown, metric, score_calculation)[source]#

Transform a list of results from Elasticsearch into a matches in the form of a DocumentArray.

  • query_doc (Document) – the query document.

  • es_results (List[Dict]) – List of dictionaries containing results from Elasticsearch querying.

  • get_score_breakdown (bool) – whether to calculate the score breakdown for matches.

  • metric (str) – the metric used to calculate the score.

  • score_calculation – the score calculation for each match.

Return type



DocumentArray that holds all matches in the form of `Document`s.

now.executor.indexer.elastic.es_converter.calculate_score_breakdown(query_doc, retrieved_doc, score_calculation, metric)[source]#

Calculate the score breakdown for a given retrieved document. Each score calculation in the indexer’s score_calculation should have a corresponding value, returned inside a list of scores in the documents tags under score_breakdown.

  • query_doc (Document) – The query document. Contains embeddings for the score calculation at tag level.

  • retrieved_doc (Document) – The Elasticsearch results, containing embeddings inside the _source field.

  • score_calculation – The score calculation used for the score breakdown.

  • metric – The metric to be used for the score breakdown.

Return type



List of integers representing the score breakdown.

now.executor.indexer.elastic.es_converter.calculate_l2_norm(d_emb, q_emb)[source]#
now.executor.indexer.elastic.es_converter.calculate_cosine(d_emb, q_emb)[source]#