U Lk7g(@sBddlZddlZddlZddlZddlmZddlZddlZddlm Z m Z ddl m Z ddl mZddlmZddlmZddlmZdd lmZdd lmZdd lmZdd lmZmZmZmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%ddl&m'Z'm(Z(ddl)m*Z*ddl+m,Z,ddl-m.Z.m/Z/dZ0dZ1dZ2dZ3dZ4dZ5GdddZ6dS)N)Mapping)SON json_util)Code)ReturnDocument)validate_read_preference) ReadConcern)signals get_document) _import_class)get_db)!no_dereferencing_active_for_classset_read_write_concernset_write_concern switch_db)BulkWriteErrorInvalidQueryError LookUpErrorNotUniqueErrorOperationError)LEGACY_JSON_OPTIONScount_documents) transform)QueryFieldList)QQNode) BaseQuerySet DO_NOTHINGNULLIFYCASCADEDENYPULLc@seZdZdZddZdddZddZd d Zd d Zd dZ ddZ ddZ ddZ ddZ dddZddZddZddZdd d!Zdd#d$Zdd%d&Zdd'd(Zdd)d*Zdd+d,Zdd-d.Zd/d0Zd1d2Zd3d4Zd5d6Zd7d8Zd9d:Zd;d<Zdd>d?Z d@dAZ!dBdCZ"ddDdEZ#ddFdGZ$dHdIZ%dJdKZ&dLdMZ'dNdOZ(ddPdQZ)dRdSZ*ddTdUdVZ+dWdXZ,dYdZZ-d[d\Z.d]d^Z/d_d`Z0dadbZ1dcddZ2dedfZ3dgdhZ4didjZ5dkdlZ6dmdnZ7dodpZ8dqdrZ9dsdtZ:ddudvZ;dwdxZd}d~Z?dddZ@ddZAddZBeCddZDeCddZEeCddZFddZGeCddZHeCddZIeCddZJddZKdddZLdddZMddZNddZOddZPddZQddZRdS)rzA set of results returned from a query. Wraps a MongoDB cursor, providing :class:`~mongoengine.Document` objects as the results. cCs||_||_d|_t|_i|_d|_t|_d|_ d|_ d|_ d|_ d|_ d|_d|_g|_d|_d|_d|_d|_d|_d|_|jddkrt|jjdkrd|jjdi|_ndd|jjii|_tdgd|_d|_d|_d|_d |_d|_d|_ d|_!d|_"d|_#dS) NFTallow_inheritancer#_clsr$inalways_include)$ _document_collection_obj _mongo_queryr _query_obj _cls_query _where_clauser_loaded_fields _ordering _snapshot_timeout_allow_disk_use_read_preference _read_concern_iter_scalar_none _as_pymongo _search_text_search_text_score_BaseQuerySet__dereference_BaseQuerySet__auto_dereference_metagetlen _subclasses _cursor_obj_limit_skip_hint _collation _batch_size _max_time_ms_comment_empty)selfdocument collectionrR=/tmp/pip-unpacked-wheel-n1etwkgt/mongoengine/queryset/base.py__init__7sFzBaseQuerySet.__init__NcKsVtf|}|r0t|ts(d|}t|||M}|}|j|M_d|_d|_|S)aFilter the selected documents by calling the :class:`~mongoengine.queryset.QuerySet` with a query. :param q_obj: a :class:`~mongoengine.queryset.Q` object to be used in the query; the :class:`~mongoengine.queryset.QuerySet` is filtered multiple times with different :class:`~mongoengine.queryset.Q` objects, only the last one will be used. :param query: Django-style query keyword arguments. z8Not a query object: %s. Did you intend to use key=value?N)r isinstancerrcloner0r/rF)rOZq_objquerymsgquerysetrRrRrS__call__is  zBaseQuerySet.__call__cCs |j}|dd|d<|S)o Need for pickling queryset See https://github.com/MongoEngine/mongoengine/issues/442 r.NrF)__dict__copypoprOZobj_dictrRrRrS __getstate__s  zBaseQuerySet.__getstate__cCs |d|d<|j|dS)r[r-r.N)_get_collectionr\updater_rRrRrS __setstate__szBaseQuerySet.__setstate__cCs|}d|_t|trd|j||_|j|j|_|_ |jrP|jrP|j|j|_ |j dkr`d|_|St|t r|j r| |j j|j||jdS|jr|j|S|j j|j||jdStddS)aReturn a document instance corresponding to a given index if the key is an integer. If the key is a slice, translate its bounds into a skip and a limit, and return a cloned queryset with that skip/limit applied. For example: >>> User.objects[0] >>> User.objects[1:3] [, ] FrT_auto_dereferencez#Provide a slice or an integer indexN)rVrNrUslice_cursorrFstartstoprHrGintr; _get_scalarr- _from_sonrer= TypeError)rOkeyrYrRrRrS __getitem__s2       zBaseQuerySet.__getitem__cCstdSN)NotImplementedErrorrOrRrRrS__iter__szBaseQuerySet.__iter__cCs|}|dkrdSdS)z#Return True if cursor has any data.NFT)order_byfirstrOrYrRrRrS _has_dataszBaseQuerySet._has_datacCs|S)z/Avoid to open all records in an if stmt in Py3.)rwrrrRrRrS__bool__szBaseQuerySet.__bool__cCs|S)z'Returns a copy of the current QuerySet.rZrrrRrRrSallszBaseQuerySet.allcOs |j||S)z;An alias of :meth:`~mongoengine.queryset.QuerySet.__call__`ry)rOq_objsrWrRrRrSfilterszBaseQuerySet.filterTcCsb|}|jrtdtd|i}|r.||d<|jtd|idM_d|_d|_||_||_|S)a Start a text search, using text indexes. Require: MongoDB server version 2.6+. :param language: The language that determines the list of stop words for the search and the rules for the stemmer and tokenizer. If not specified, the search uses the default language of the index. For supported languages, see `Text Search Languages `. :param text_score: True to have it return the text_score (available through get_text_score()), False to disable that Note that unless you order the results, leaving text_score=True may provide randomness in the returned documents z0It is not possible to use search_text two times.z$searchz $languagez$text__raw__N) rVr>rrr0rr/rFr?)rOtextlanguageZ text_scorerYZ query_kwargsrRrRrS search_texts  zBaseQuerySet.search_textcOs|}|d}|j||}z t|}Wn,tk rZd|jj}|j|YnXz t|Wntk r|YSX|j ddS)a4Retrieve the matching object raising :class:`~mongoengine.queryset.MultipleObjectsReturned` or `DocumentName.MultipleObjectsReturned` exception if multiple results and :class:`~mongoengine.queryset.DoesNotExist` or `DocumentName.DoesNotExist` if no results are found. r$z!%s matching query does not exist.z&2 or more items returned, instead of 1N) rVrtlimitr|next StopIterationr- _class_nameZ DoesNotExistZMultipleObjectsReturned)rOr{rWrYresultrXrRrRrSrCs     zBaseQuerySet.getcKs|jf|jddS)z5Create new object. Returns the saved object instance.T)Z force_insert)r-save)rOkwargsrRrRrScreate"szBaseQuerySet.createcCsB|}|js|jrdSz |d}Wntk r<d}YnX|S)z-Retrieve the first object matching the query.Nr)rVr<rN IndexError)rOrYrrRrRrSru&s   zBaseQuerySet.firstc sbtd}|dkri}|}d}t||s2t|j|r}t||jsfdt|j} t| |jr@|js@d} t| q@|pi}t j j |jfd|i|dd |D} t |j |} | j} |r| d } | j} W5QRXz| | } |r| jgn| j}Wntjjk r6}zd }t||W5d}~XYntjjk rn}zd }t||jW5d}~XYnZtjjk r}z6d }td t|rd}t||t||W5d}~XYnXt||D]\}}||_q|st jj |jf|dd||r|d S|S||fdd |D}t jj |jf|dd||r^|d S|S)ambulk insert documents :param doc_or_docs: a document or list of documents to be inserted :param load_bulk (optional): If True returns the list of document instances :param write_concern: Extra keyword arguments are passed down to :meth:`~pymongo.collection.Collection.insert` which will be used as options for the resultant ``getLastError`` command. For example, ``insert(..., {w: 2, fsync: True})`` will wait until at least two servers have recorded the write and will force an fsync on each server being written to. :param signal_kwargs: (optional) kwargs dictionary to be passed to the signal calls. By default returns document instances, set ``load_bulk`` to False to return just ``ObjectIds`` DocumentNFTz.Some documents inserted aren't instances of %sz7Some documents have ObjectIds, use doc.update() instead documentscSsg|] }|qSrR)Zto_mongo.0docrRrRrS _sz'BaseQuerySet.insert..rzCould not save document (%s)zBulk write error: (%s)z^E1100[01] duplicate keyz(Tried to save duplicate unique keys (%s))rZloadedcsg|]}|qSrR)rC)rZobj_idrrRrSrs)r rU issubclass __class__r-strrpkZ_createdr Zpre_bulk_insertsendr _collectionZ insert_manyZ insert_oneZ inserted_idZ inserted_idspymongoerrorsDuplicateKeyErrorrrdetailsOperationFailurerematchzipZpost_bulk_insertin_bulk)rOZ doc_or_docsZ load_bulk write_concernZ signal_kwargsrdocsZ return_onerrXrawrQZ insert_funcZinserted_resultidserrmessageZdoc_idresultsrRrrSinsert2s       zBaseQuerySet.insertFcCs|jdkr|dks|js|jr"dS|r4|j|jdni}|jdkrN|dd|jdkrb|j|d<|jrr|j|d<tf|jj |j d |}d|_ |S) zCount the selected elements in the query. :param with_limit_and_skip (optional): take any :meth:`limit` or :meth:`skip` that has been applied to this cursor into account when getting the count rF)rskiprN)r,Nhint collation)rQr|) rGr<rNrHr^rIrJrrgrQ_queryrF)rOZwith_limit_and_skiprcountrRrRrSrs2      zBaseQuerySet.countc CsH|}|j}|dkri}tjo6tj|p6tj|}|jsH|jsH|oL| }|rxd}|D]}|j f||d7}qZ|S|j dpi} t | } | D]^\} } | \} } | j drq| tkr| jf| d|i}|ddkrtd| j| fq| D]\} } | \} } | j drq| tkr|dkr8tn|}|j| jkrb|D]}||jqN| jf| d|d|i}|dkr|j ||d q| tkr| jf| d|ijfd |id | diq| tkr| jf| d|ijfd |id | |iqt|j|,}||j}|jr:|j W5QRSW5QRXdS) akDelete the documents matched by the query. :param write_concern: Extra keyword arguments are passed down which will be used as options for the resultant ``getLastError`` command. For example, ``save(..., write_concern={w: 2, fsync: True}, ...)`` will wait until at least two servers have recorded the write and will force an fsync on the primary server. :param _from_doc_delete: True when called from document delete therefore signals will have been triggered so don't loop. :returns number of deleted documents Nrr# delete_rulesZabstractZ__inz.Could not delete document (%s.%s refers to it)Zpk__nin)r cascade_refsrz unset__%sz pull_all__%s)!rVr-r Zsignals_availableZ pre_deleteZhas_receivers_forZ post_deleterHrGdeleterBrClistitemsr!objectsrrr__name__r setraddidrrbr"rZ delete_manyrZ acknowledgedZ deleted_count)rOrZ_from_doc_deleterrYrZhas_delete_signalZcall_document_deleteZcntrZ rule_entryZruleZ document_cls field_namerefsrefrQrrRrRrSrs|            zBaseQuerySet.deletec s|s|std|dkri}|js(|jr,dS|j}d|krht|dtrhfdd|dD}ntjj f|}|rd|krd|krj j |dd<ndj j i|d<z\t j ||&} | j } |r| j} | ||||d } W5QRX|r| WS| jr| jd WSWntjjk r>} ztd | W5d} ~ XYnNtjjk r} z*t| d krnd } t| td | W5d} ~ XYnXdS)aPerform an atomic update on the fields matched by the query. :param upsert: insert if document doesn't exist (default ``False``) :param multi: Update multiple documents. :param write_concern: Extra keyword arguments are passed down which will be used as options for the resultant ``getLastError`` command. For example, ``save(..., write_concern={w: 2, fsync: True}, ...)`` will wait until at least two servers have recorded the write and will force an fsync on the primary server. :param read_concern: Override the read concern for the operation :param full_result: Return the associated ``pymongo.UpdateResult`` rather than just the number updated items :param array_filters: A list of filters specifying which array elements an update should apply. :param update: Django-style update keyword arguments :returns the number of updated documents (unless ``full_result`` is True) z'No update parameters, would remove dataNrr~cs g|]}tjjfd|iqSr})rrbr-)rurYrRrSr<sz'BaseQuerySet.update..r(z$set)upsert array_filtersnUpdate failed (%s)zmulti not coded yetz'update() method requires MongoDB 1.1.3+)rr<rNrVrrUrrrbr-rrr update_oneZ update_many raw_resultrrrrrr)rOrmultir read_concern full_resultrrbrWrQZ update_funcrrrrRrrSrbs^    zBaseQuerySet.updatecKsD|jfdd||dd|}|jdr0|}n|jj|j}|S)awOverwrite or add the first document matched by the query. :param write_concern: Extra keyword arguments are passed down which will be used as options for the resultant ``getLastError`` command. For example, ``save(..., write_concern={w: 2, fsync: True}, ...)`` will wait until at least two servers have recorded the write and will force an fsync on the primary server. :param read_concern: Override the read concern for the operation :param update: Django-style update keyword arguments :returns the new or overwritten document FT)rrrrrZupdatedExisting)rbrrCr-rwith_idZ upserted_id)rOrrrbZ atomic_updaterPrRrRrS upsert_one_s  zBaseQuerySet.upsert_onecKs|jf|d|||d|S)aPerform an atomic update on the fields of the first document matched by the query. :param upsert: insert if document doesn't exist (default ``False``) :param write_concern: Extra keyword arguments are passed down which will be used as options for the resultant ``getLastError`` command. For example, ``save(..., write_concern={w: 2, fsync: True}, ...)`` will wait until at least two servers have recorded the write and will force an fsync on the primary server. :param full_result: Return the associated ``pymongo.UpdateResult`` rather than just the number updated items :param array_filters: A list of filters specifying which array elements an update should apply. :param update: Django-style update keyword arguments full_result :returns the number of updated documents (unless ``full_result`` is True) F)rrrrr)rb)rOrrrrrbrRrRrSr}szBaseQuerySet.update_onec Ksr|r|rtd|s$|s$|s$td|js0|jr4dS|}|j}|sVtj|jf|}|j} zl|rrd} t | t |r|j j |fd| i|j} n4|rtj} ntj} |j j||f|| | |d|j} Wndtjjk r} ztd| W5d} ~ XYn4tjjk r,} ztd| W5d} ~ XYnX|rX| ddk rn|j| d| d<n| dk rn|j| } | S) aUpdate and return the updated document. Returns either the document before or after modification based on `new` parameter. If no documents match the query and `upsert` is false, returns ``None``. If upserting and `new` is false, returns ``None``. If the full_response parameter is ``True``, the return value will be the entire response object from the server, including the 'ok' and 'lastErrorObject' fields, rather than just the modified document. This is useful mainly because the 'lastErrorObject' document holds information about the command's execution. :param upsert: insert if document doesn't exist (default ``False``) :param full_response: return the entire response object from the server (default ``False``, not available for PyMongo 3+) :param remove: remove rather than updating (default ``False``) :param new: return updated rather than original document (default ``False``) :param array_filters: A list of filters specifying which array elements an update should apply. :param update: Django-style update keyword arguments z&Conflicting parameters: remove and newz2No update parameters, must either update or removeNzEWith PyMongo 3+, it is not possible anymore to get the full response.sort)rrZreturn_documentrrvalue)rr<rNrVrrrbr-r4warningswarnDeprecationWarningrZfind_one_and_delete _cursor_argsrZAFTERZBEFOREZfind_one_and_updaterrrrrrl)rOrZ full_responseremovenewrrbrYrWrrXrZ return_docrrRrRrSmodifys^      zBaseQuerySet.modifycCs*|}|jrd}t||j|dS)aRetrieve the object matching the id provided. Uses `object_id` only and raises InvalidQueryError if a filter has been applied. Returns `None` if no document exists with that id. :param object_id: the value for the id of the document to look up z*Cannot use a filter whilst using `with_id`)r)rVr0rr|ru)rOZ object_idrYrXrRrRrSrs zBaseQuerySet.with_idcCsi}|jjdd|iif|j}|jrJ|D]}||j|||d<q(nB|jrh|D]}|||d<qTn$|D]}|jj||jd||d<ql|S)zRetrieve a set of documents by their ids. :param object_ids: a list or tuple of ObjectId's :rtype: dict of ObjectId's as keys and collection-specific Document subclasses as values. _idr)rd) rfindrr;rkr-rlr=re)rOZ object_idsZdoc_maprrrRrRrSrszBaseQuerySet.in_bulkcCs|}d|_|S)zReturns a queryset that never returns any objects and no query will be executed when accessing the results inspired by django none() https://docs.djangoproject.com/en/dev/ref/models/querysets/#none T)rVr<rvrRrRrSnoneszBaseQuerySet.nonecCs$|jjddkr d|jji|_|S)zqFilter for only the instances of this specific document. Do NOT return any inherited documents. r'Tr()r-rBrCrr1rrrRrRrSno_sub_classesszBaseQuerySet.no_sub_classesc Cs4t|j|}|}W5QRX|||j|S)zThis method is for controlling which database the QuerySet will be evaluated against if you are using more than one database. :param alias: The database alias )rr-ra _clone_intor)rOaliasclsrQrRrRrSusing)szBaseQuerySet.usingcCs|||j|jS)z&Create a copy of the current queryset.)rrr-r.rrrRrRrSrV5szBaseQuerySet.clonecCs`t|tstd|jd}|D] }t||}t||t|q |j|_|jr\|j |_|S)zCopy all the relevant properties of this queryset to a new queryset (which has to be an instance of :class:`~mongoengine.queryset.base.BaseQuerySet`). z$%s is not a subclass of BaseQuerySet)r/r1r<r0r2r3r4r5r6r7r8r9r:r;r=rGrHrNrIrJr>r?rLrMrK) rUrrrgetattrsetattrr]rArFrV)rOZnew_qsZ copy_propspropvalrRrRrSr9s   zBaseQuerySet._clone_intor#cCs|d7}|}|j||dS)zHandles dereferencing of :class:`~bson.dbref.DBRef` objects or :class:`~bson.object_id.ObjectId` a maximum depth in order to cut down the number queries to mongodb. r#) max_depth)rV _dereference)rOrrYrRrRrSselect_relatedjszBaseQuerySet.select_relatedcCs,|}||_d|_|jr(|j|j|S)a&Limit the number of returned documents to `n`. This may also be achieved using array-slicing syntax (e.g. ``User.objects[:5]``). :param n: the maximum number of objects to return if n is greater than 0. When 0 is passed, returns all the documents in the cursor F)rVrGrNrFrrOrrYrRrRrSrts zBaseQuerySet.limitcCs&|}||_|jr"|j|j|S)zSkip `n` documents before returning the results. This may also be achieved using array-slicing syntax (e.g. ``User.objects[5:]``). :param n: the number of objects to skip before returning results )rVrHrFrrrRrRrSrs zBaseQuerySet.skipcCs&|}||_|jr"|j|j|S)aAdded 'hint' support, telling Mongo the proper index to use for the query. Judicious use of hints can greatly improve query performance. When doing a query on multiple fields (at least one of which is indexed) pass the indexed field as a hint to the query. Hinting will not do anything if the corresponding index does not exist. The last hint applied to this cursor takes precedence over all others. )rVrIrFr)rOindexrYrRrRrSrs zBaseQuerySet.hintcCs$|}||_|jr |j||S)a\ Collation allows users to specify language-specific rules for string comparison, such as rules for lettercase and accent marks. :param collation: `~pymongo.collation.Collation` or dict with following fields: { locale: str, caseLevel: bool, caseFirst: str, strength: int, numericOrdering: bool, alternate: str, maxVariable: str, backwards: str } Collation should be added to indexes like in test example )rVrJrFr)rOrrYrRrRrSrs  zBaseQuerySet.collationcCs&|}||_|jr"|j|j|S)a&Limit the number of documents returned in a single batch (each batch requires a round trip to the server). See https://pymongo.readthedocs.io/en/stable/api/pymongo/cursor.html#pymongo.cursor.Cursor for details. :param size: desired size of each batch. )rVrKrF batch_size)rOsizerYrRrRrSrs zBaseQuerySet.batch_sizec sd|}z||g}Wntk r0YnX|j|}|jsH|S|j|d||jd}|jj | ddd}dt d}t d}t d}t ||rt|d |}t |||frt|d dd|kr8| dddD]X} rt |||fr}t|| |}t ||rt|d |}t |||frt|d dqވr`t |||fr`fd d |D}|S) zReturn a list of distinct values for a given field. :param field: the field to select distinct values from .. note:: This is a command and won't take ordering or limit into account. r#)nameinstance.rNEmbeddedDocumentField ListFieldGenericEmbeddedDocumentFieldfieldZ document_typecsg|]}f|qSrRrRrrrRrSr sz)BaseQuerySet.distinct..)rV_fields_to_dbfieldsr^rrgdistinctrerr-_fieldsrCsplitr rUr) rOrrYZ raw_valuesrZ doc_fieldrrrZ field_partrRrrSrsN        zBaseQuerySet.distinctcGsdd|D}|jd|S)aLoad only a subset of this document's fields. :: post = BlogPost.objects(...).only('title', 'author.name') .. note :: `only()` is chainable and will perform a union :: So with the following it will fetch both: `title` and `author.name`:: post = BlogPost.objects.only('title').only('author.name') :func:`~mongoengine.queryset.QuerySet.all_fields` will reset any field filters. :param fields: fields to include cSsi|] }|tjqSrR)rZONLYrfrRrRrS sz%BaseQuerySet.only..T)TfieldsrOrrRrRrSonlyszBaseQuerySet.onlycGsdd|D}|jf|S)aOpposite to .only(), exclude some document's fields. :: post = BlogPost.objects(...).exclude('comments') .. note :: `exclude()` is chainable and will perform a union :: So with the following it will exclude both: `title` and `author.name`:: post = BlogPost.objects.exclude('title').exclude('author.name') :func:`~mongoengine.queryset.QuerySet.all_fields` will reset any field filters. :param fields: fields to exclude cSsi|] }|tjqSrR)rZEXCLUDErrRrRrSr/sz(BaseQuerySet.exclude..rrrRrRrSexclude szBaseQuerySet.excludec Ksddg}g}|D]L\}}|d}|d|krH|d}d||i}d|}|||fqdd} t|| d } |} t| d d D]8\}} d d | D} | | } | j t | ||d7_ q| S)a[Manipulate how you load this document's fields. Used by `.only()` and `.exclude()` to manipulate which fields to retrieve. If called directly, use a set of kwargs similar to the MongoDB projection document. For example: Include only a subset of fields: posts = BlogPost.objects(...).fields(author=1, title=1) Exclude a specific field: posts = BlogPost.objects(...).fields(comments=0) To retrieve a subrange or sublist of array elements, support exist for both the `slice` and `elemMatch` projection operator: posts = BlogPost.objects(...).fields(slice__comments=5) posts = BlogPost.objects(...).fields(elemMatch__comments="test") :param kwargs: A set of keyword arguments identifying what to include, exclude, or slice. rfZ elemMatch__r$rcSs|\}}t|tr|SdS)Nr$)rUrj)Z field_tuple_rrRrRrS _sort_keyXs z&BaseQuerySet.fields.._sort_key)rncSs|dS)Nr#rR)xrRrRrScz%BaseQuerySet.fields..cSsg|] \}}|qSrRrR)rrrrRrRrSrdsz'BaseQuerySet.fields..)r _only_called) rrr^joinappendsortedrV itertoolsgroupbyrr3r) rOrr operatorsZcleaned_fieldsrnrpartsoprrrYgrouprRrRrSr2s*        zBaseQuerySet.fieldscCs|}t|jjd|_|S)zInclude all fields. Reset all previously calls of .only() or .exclude(). :: post = BlogPost.objects.exclude('comments').all_fields() r*)rVrr3r+rvrRrRrS all_fieldsls zBaseQuerySet.all_fieldsr}cGs^|r|rtd|}|j}|r(|}n ||}|jrT|rJ|j|n |rTd|_||_|S)aOrder the :class:`~mongoengine.queryset.QuerySet` by the given keys. The order may be specified by prepending each of the keys by a "+" or a "-". Ascending order is assumed if there's no prefix. If no keys are passed, existing ordering is cleared instead. :param keys: fields to order the query results by; keys may be prefixed with "+" or a "-" to determine the ordering direction. :param __raw__: a raw pymongo "sort" argument (provided as a list of (key, direction)) see 'key_or_list' in `pymongo.cursor.Cursor.sort doc `. If both keys and __raw__ are provided, an exception is raised z2Can not use both keys and __raw__ with order_by() N)rrVr4 _get_order_byrFr)rOr~keysrYZ old_orderingZ new_orderingrRrRrSrtxs zBaseQuerySet.order_bycCs|}i|_|S)abClear the default "_cls" query. By default, all queries generated for documents that allow inheritance include an extra "_cls" clause. In most cases this is desirable, but sometimes you might achieve better performance if you clear that default query. Scan the code for `_cls_query` to get more details. )rVr1rvrRrRrSclear_cls_querys zBaseQuerySet.clear_cls_querycCs |d|S)zAdd a comment to the query. See https://www.mongodb.com/docs/manual/reference/method/cursor.comment/ for details. comment_chainable_method)rOrrRrRrSr szBaseQuerySet.commentcCs |jS)zfReturn an explain plan record for the :class:`~mongoengine.queryset.QuerySet` cursor. )rgexplainrrrRrRrSrszBaseQuerySet.explaincCs"d}t|t|}||_|S)zxEnable or disable snapshot mode when querying. :param enabled: whether or not snapshot mode is enabled zAsnapshot is deprecated as it has no impact when using PyMongo 3+.)rrrrVr5)rOenabledrXrYrRrRrSsnapshots  zBaseQuerySet.snapshotcCs|}||_|S)zEnable or disable the use of temporary files on disk while processing a blocking sort operation. (To store data exceeding the 100 megabyte system memory limit) :param enabled: whether or not temporary files on disk are used )rVr7rOrrYrRrRrSallow_disk_useszBaseQuerySet.allow_disk_usecCs|}||_|S)zEnable or disable the default mongod timeout when querying. (no_cursor_timeout option) :param enabled: whether or not the timeout is used )rVr6rrRrRrStimeoutszBaseQuerySet.timeoutcCs"td||}||_d|_|S)zChange the read_preference when querying. :param read_preference: override ReplicaSetConnection-level preference. read_preferenceN)rrVr8rF)rOrrYrRrRrSrs  zBaseQuerySet.read_preferencecCsJ|dk r t|ts t|d|}|dk r:tf|nd|_d|_|S)zChange the read_concern when querying. :param read_concern: override ReplicaSetConnection-level preference. Nz is not a valid read concern.)rUrrmrVrr9rF)rOrrYrRrRrSrszBaseQuerySet.read_concerncGs.|}t||_|r"|j|}n|}|S)aInstead of returning Document instances, return either a specific value or a tuple of values in order. Can be used along with :func:`~mongoengine.queryset.QuerySet.no_dereference` to turn off dereferencing. .. note:: This effects all results and can be unset by calling ``scalar`` without arguments. Calls ``only`` automatically. :param fields: One or more fields to return instead of a Document. )rVrr;rr )rOrrYrRrRrSscalars   zBaseQuerySet.scalarcGs |j|S)zAn alias for scalar)rrrRrRrS values_list szBaseQuerySet.values_listcCs|}d|_|S)zInstead of returning Document instances, return raw values from pymongo. This method is particularly useful if you don't need dereferencing and care primarily about the speed of data retrieval. T)rVr=rvrRrRrS as_pymongoszBaseQuerySet.as_pymongocCs |d|S)zWait `ms` milliseconds before killing the query on the server :param ms: the number of milliseconds before killing the query on the server max_time_msr)rOmsrRrRrSrszBaseQuerySet.max_time_mscOs2d|krtdtt|d<tj|f||S)zConverts a queryset to JSONZ json_optionsaNo 'json_options' are specified! Falling back to LEGACY_JSON_OPTIONS with uuid_representation=PYTHON_LEGACY. For use with other MongoDB drivers specify the UUID representation to use. This will be changed to uuid_representation=UNSPECIFIED in a future release.)rrrrrdumpsr)rOargsrrRrRrSto_json%szBaseQuerySet.to_jsoncst|}fdd|DS)z%Converts json data to unsaved objectscsg|]}j|qSrR)r-rl)rdatarrrRrSr6sz*BaseQuerySet.from_json..)rloads)rOZ json_dataZson_datarRrrrS from_json3s zBaseQuerySet.from_jsonc Os4t|tpt|}t|tr"|gnt|}|r>d}t|t||7}g}|jsV|jrv| ddi| dddii|j r| d|j i|j r| dt|j i|j dk r| d|j |j pd i|j dk r| d |j i||}|j} |jdk s |jdk r|jj|j|jd } | j|fd ii|S) aAPerform an aggregate function based on your queryset params :param pipeline: list of aggregation commands, see: https://www.mongodb.com/docs/manual/core/aggregation-pipeline/ :param suppl_pipeline: unpacked list of pipeline (added to support deprecation of the old interface) parameter will be removed shortly :param kwargs: (optional) kwargs dictionary to be passed to pymongo's aggregate call See https://pymongo.readthedocs.io/en/stable/api/pymongo/collection.html#pymongo.collection.Collection.aggregate zCalling .aggregate() with un unpacked list (*pipeline) is deprecated, it will soon change and will expect a list (similar to pymongo.Collection.aggregate interface), see documentationz$limitr#$matchz$exprFz$sortNrz$skiprrcursor)rUdictboolrrrrr<rNrrr4rGrHrr8r9 with_options aggregate) rOpipelineZsuppl_pipelinerZusing_deprecated_interfaceZ user_pipelinerXZinitial_pipelineZfinal_pipelinerQrRrRrSr(8s4     zBaseQuerySet.aggregateccs||}td}i} t|tr,|j} t|}t||| p.)rQoptionsz$whererW)r,eval)rretval) rVr5r-r8rr2rr6r7rC)rOcoderr?rQr,rWr3rRrrSexec_jss     zBaseQuerySet.exec_jscCs|}||}||_|S)aFilter ``QuerySet`` results with a ``$where`` clause (a Javascript expression). Performs automatic field name substitution like :meth:`mongoengine.queryset.Queryset.exec_js`. .. note:: When using this mode of query, the database will call your function, or evaluate your predicate clause, for each object in the collection. )rVr5r2)rO where_clauserYrRrRrSwheres  zBaseQuerySet.wherecCs||g}d|jidddd|idig}td}|d}|j|}t|d |rp|d d d|it |j |}|r|d d Sd S)zSum over the values of the specified field. :param field: the field to sum over; use dot notation to refer to embedded document fields r"$groupsumz$sumrrtotalrrr,r#$unwindrrI rr^rr rr- _lookup_fieldrUrtuplerar(rOrdb_fieldr)r field_partsZfield_instancesrrRrRrSrGs   zBaseQuerySet.sumcCs||g}d|jidddd|idig}td}|d}|j|}t|d |rp|d d d|it |j |}|r|d d Sd S)zAverage over the values of the specified field. :param field: the field to average over; use dot notation to refer to embedded document fields r"rFavgz$avgrrHrrr,r#rJrrIrKrNrRrRrSaverage6s   zBaseQuerySet.averagecCs |r|j||dS|j||dS)a"Returns a dictionary of all items present in a field across the whole queried set of documents, and their corresponding frequency. This is useful for generating tag clouds, or searching documents. .. note:: Can only do direct simple mappings and cannot map across :class:`~mongoengine.fields.ReferenceField` or :class:`~mongoengine.fields.GenericReferenceField` for more complex counting a manual map reduce call is required. If the field is a :class:`~mongoengine.fields.ListField`, the items within each list will be counted individually. :param field: the field to use :param normalize: normalize the results so they add to 1.0 :param map_reduce: Use map_reduce over exec_js ) normalize)_item_frequencies_map_reduce_item_frequencies_exec_js)rOrrSr=rRrRrSitem_frequenciesPszBaseQuerySet.item_frequenciescCsJ|js |jrtt|j}|jr$|S|jj||jd}|j rF| |S|S)z;Wrap the result in a :class:`~mongoengine.Document` object.rd) r<rNrrrgr=r-rlrer;rk)rOZraw_docrrRrRrS__next__is   zBaseQuerySet.__next__cCsd|_|jdS)z+Rewind the cursor to its unevaluated state.FN)r:rgrewindrrrRrRrSrX}szBaseQuerySet.rewindcCs|jS)zProperty that returns the collection object. This allows us to perform operations only if the collection is accessed. )r.rrrRrRrSrszBaseQuerySet._collectioncCs~d}|jrd}t|ti}|js,d|d<|jr:d|d<|jrN|j||<|jrz||krdi||<|j rzddi||d<|S) NZ projectionzr?)rOZ fields_namerXZ cursor_argsrRrRrSrs" zBaseQuerySet._cursor_argscCsf|jdk r|jS|jdk s$|jdk rJ|jj|j|jdj|jf|j|_n|jj|jf|j|_|jr~| |j}|j ||j r|j |j n4|j dkr|j jdr||j jd}|j ||jdk r|j|j|jdk r|j|j|jdkr|j|j|jdk r,|j|j|jdk rF|j|j|jdk r`|j|j|jS)z>Return a PyMongo cursor object corresponding to this queryset.Nr#Zorderingr,)rFr8r9rr'rrrr2r5rEr4rr-rBr rGrrHrrIrrJrrKrrMr )rOrDorderrRrRrSrgsB          zBaseQuerySet._cursorcCs|S)z;Essential for chained queries with ReferenceFields involved)rV)rOmemorRrRrS __deepcopy__szBaseQuerySet.__deepcopy__cCsR|jdkrL|j|j|_|jrLd|jkr>d|j|jgi|_n|j|j|jS)Nr(z$and)r/r0Zto_queryr-r1rbrrrRrRrSrs  zBaseQuerySet._querycCs|jstd|_|jS)NZ DeReference)r@r rrrRrRrSrs zBaseQuerySet._dereferencecCst|j }|o|jSrp)rr-rA)rOZ should_derefrRrRrSres zBaseQuerySet._auto_dereferencecCs|}d|_|S)z.) formatr=rnrUrarjrrGvaluesr) rOrrSZmap_funcZ reduce_funcrg frequenciesrrnrRrerSrTs     z)BaseQuerySet._item_frequencies_map_reducec sd}|||\}fdd|D}|rFfdd|D}i}|D].\}}t|trxt||krxt|}|||<qR|S)zUses exec_js to executeaa function(path) { var path = path.split('.'); var total = 0.0; db[collection].find(query).forEach(function(doc) { var field = doc; for (p in path) { if (field) field = field[path[p]]; else break; } if (field && field.constructor == Array) { total += field.length; } else { total++; } }); var frequencies = {}; var types = {}; var inc = 1.0; db[collection].find(query).forEach(function(doc) { field = doc; for (p in path) { if (field) field = field[path[p]]; else break; } if (field && field.constructor == Array) { field.forEach(function(item) { frequencies[item] = inc + (isNaN(frequencies[item]) ? 0: frequencies[item]); }); } else { var item = field; types[item] = item; frequencies[item] = inc + (isNaN(frequencies[item]) ? 0: frequencies[item]); } }); return [total, frequencies, types]; } cs i|]\}}|t|qSrR)rCrjrb)typesrRrSrfsz:BaseQuerySet._item_frequencies_exec_js..csi|]\}}|t|qSrRr`rb)rIrRrSris)rCrrUrarj) rOrrSZ freq_funcrrgrhrcrdrR)rIrirSrU6s-   z&BaseQuerySet._item_frequencies_exec_jsc Csg}|jjdr*dd|jjDdd}g}|D]}|d}z*ddd|j|D}||Wq2tk r}zbd }|D]L}z2dd d||D} || d }WqWqtk rYqXq|s|W5d}~XYq2Xq2|S) z0Translate fields' paths to their db equivalents.r'cSsg|] }t|qSrRr )rrrRrRrSrysz4BaseQuerySet._fields_to_dbfields..r#Nrcss"|]}t|tr|n|jVqdSrprUrrOrrRrRrS sz3BaseQuerySet._fields_to_dbfields..Fcss"|]}t|tr|n|jVqdSrprjrrRrRrSrksT)r-rBrErrrLrr) rOrZ subclassesZdb_field_pathsrrPrfoundZsubdocZsubfieldrRrRrSrus4       z BaseQuerySet._fields_to_dbfieldsc Csg}|D]}|sq|dkr.|dddifqtj}|ddkrFtj}|ddkr^|dd }|d d }z|j|}Wntk rYnX|||fq|S) aGiven a list of MongoEngine-style sort keys, return a list of sorting tuples that can be applied to a PyMongo cursor. For example: >>> qs._get_order_by(['-last_name', 'first_name']) [('last_name', -1), ('first_name', 1)] z $text_scorer[rYrZr-)rm+r#Nrr)rrZ ASCENDINGZ DESCENDINGr/r-r> Exception)rOr Zkey_listrn directionrRrRrSr s&    zBaseQuerySet._get_order_bycs:ddfdd|jD}t|dkr2|dSt|S)NcSs"|d}|D]}t||}q|S)Nr)rr)objrchunkschunkrRrRrSlookups  z(BaseQuerySet._get_scalar..lookupcsg|]}|qSrRrR)rrrrtrRrSrsz,BaseQuerySet._get_scalar..r#r)r;rDrM)rOrrrRrurSrks  zBaseQuerySet._get_scalarcs8fdd}fdd}td||}td||}|S)aWhen fields are specified with [~fieldname] syntax, where *fieldname* is the Python name of a field, *fieldname* will be substituted for the MongoDB name of the field (specified using the :attr:`name` keyword argument in a field's constructor). cs*|dd}j|}d|djS)Nr#rz["%s"]r,)rrr-rLrOrrrrrrRrS field_subs z.BaseQuerySet._sub_js_fields..field_subcs0|dd}j|}ddd|DS)Nr#rcSsg|] }|jqSrR)rOrrRrRrSrszGBaseQuerySet._sub_js_fields..field_path_sub..)rrr-rLrrvrrrRrSfield_path_subs z3BaseQuerySet._sub_js_fields..field_path_subz\[\s*~([A-z_][A-z_0-9.]+?)\s*\]z#\{\{\s*~([A-z_][A-z_0-9.]+?)\s*\}\})rsub)rOrBrwrxrRrrrSr5s   zBaseQuerySet._sub_js_fieldscCs0|}|j}t|||t|d|||S)z{Call a particular method on the PyMongo cursor call a particular chainable method with the provided value. r)rVrgrr)rO method_namerrYr$rRrRrSrs zBaseQuerySet._chainable_method)N)NT)TNN)F)NFN)FTNNFN)NN)FNFN)FFFFN)r#)N)N)F)NNN)FT)F)F)Sr __module__ __qualname____doc__rTrZr`rcrorsrwrxrzr|rrCrrurrrrbrrrrrrrrrVrrrrrrrrrrrr rtr r rrrrrrrrrrrr!r(r=rCrErGrRrVrWrXpropertyrrrgr^rrrer_rTrUrr rkr5rrRrRrRrSr2s2 .  ^ ( ] L  $ R   1   : : '      4 z)    :    2 ?%# r)7r]rrrcollections.abcrrZpymongo.errorsZbsonrrZ bson.coderZpymongo.collectionrZpymongo.commonrZpymongo.read_concernrZ mongoenginer Zmongoengine.baser Zmongoengine.commonr Zmongoengine.connectionr Zmongoengine.context_managersrrrrZmongoengine.errorsrrrrrZmongoengine.pymongo_supportrrZmongoengine.querysetrZmongoengine.queryset.field_listrZmongoengine.queryset.visitorrr__all__rrr r!r"rrRrRrRrSs8