now.executor.indexer.elastic.es_converter module#
- now.executor.indexer.elastic.es_converter.convert_es_to_da(result, get_score_breakdown)[source]#
Transform Elasticsearch documents into DocumentArray. Assumes that all Elasticsearch documents have a ‘text’ field. It returns embeddings as part of the tags for each field that is encoded.
- Parameters
result (
Union[Dict,List[Dict]]) – results from an Elasticsearch query.get_score_breakdown (
bool) – whether to return the embeddings as tags for each document.
- Return type
DocumentArray- Returns
a DocumentArray containing all results.
- now.executor.indexer.elastic.es_converter.convert_doc_map_to_es(docs_map, index_name, encoder_to_fields)[source]#
Transform a dictionary (mapping encoder to DocumentArray) into a list of Elasticsearch documents. The docs_map dictionary is expected to have the following structure: {
‘encoder1’: DocumentArray([…]), ‘encoder2’: DocumentArray([…]), # same number of documents as encoder1 …
}
- Parameters
docs_map (
Dict[str,DocumentArray]) – dictionary mapping encoder to DocumentArray.index_name (
str) – name of the index to be used in Elasticsearch.encoder_to_fields (
dict) – dictionary mapping encoder to fields.
- Return type
List[Dict]- Returns
a list of Elasticsearch documents as dictionaries ready to be indexed.
- now.executor.indexer.elastic.es_converter.get_base_es_doc(doc, index_name)[source]#
- Return type
Dict
- now.executor.indexer.elastic.es_converter.convert_es_results_to_matches(query_doc, es_results, get_score_breakdown, metric, score_calculation)[source]#
Transform a list of results from Elasticsearch into a matches in the form of a DocumentArray.
- Parameters
query_doc (
Document) – the query document.es_results (
List[Dict]) – List of dictionaries containing results from Elasticsearch querying.get_score_breakdown (
bool) – whether to calculate the score breakdown for matches.metric (
str) – the metric used to calculate the score.score_calculation – the score calculation for each match.
- Return type
DocumentArray- Returns
DocumentArray that holds all matches in the form of `Document`s.
- now.executor.indexer.elastic.es_converter.calculate_score_breakdown(query_doc, retrieved_doc, score_calculation, metric)[source]#
Calculate the score breakdown for a given retrieved document. Each score calculation in the indexer’s score_calculation should have a corresponding value, returned inside a list of scores in the documents tags under score_breakdown.
- Parameters
query_doc (
Document) – The query document. Contains embeddings for the score calculation at tag level.retrieved_doc (
Document) – The Elasticsearch results, containing embeddings inside the _source field.score_calculation – The score calculation used for the score breakdown.
metric – The metric to be used for the score breakdown.
- Return type
Document- Returns
List of integers representing the score breakdown.
