/ Azure

Cosmos DB, Node.js, continuation tokens

I really enjoy the schemaless CosmosDB database, hassle-free, fast and really easy to get started.

However I got in trouble when I got back large amounts of documents, RU's went trough the roof and the application got capped really often because of this one query. It took me frustratingly long to get a proper answer to use continuation tokens however when finding this Github gist by ryancrawcour, the struggle was over. So hereby my slightly changed version:

//TODO: Set you propper COSMOSDB_COLLECTION_LINK and COSMOS_CLIENT
 queryDocumentList(collectionLink, querySpec, options) {
        return new Promise((resolve, reject) => {
            function executeNextWithRetry(iterator, callback) {
                if (typeof iterator.executeNext !== "function") {
                    return callback('err', null)
                }
                iterator.executeNext(function (err, results, responseHeaders) {
                    if (err && err.code === 429 && responseHeaders['x-ms-retry-after-ms']) {
                        // console.log("Retrying after " + responseHeaders['x-ms-retry-after-ms']);
                        setTimeout(function () {
                            executeNextWithRetry(iterator, callback);
                        }, responseHeaders['x-ms-retry-after-ms']);
                    }
                    else if (err) {
                        return callback(err, null)
                    }
                    else {
                        documents = documents.concat(results)
                        if (iterator.hasMoreResults()) {
                            executeNextWithRetry(iterator, callback)
                        }
                        else {
                            callback()
                        }
                    }
                });
            }
            if (!options) {
                options = {
                    maxItemCount: 50,
                };
            }
            
            let documents = []
            let iterator = COSMOS_CLIENT.queryDocuments(collectionLink, querySpec, options)
            executeNextWithRetry(iterator, function (err, result) {
                if (err) {
                    reject(err)
                }
                else {
                    resolve(documents)
                }
            });
        })
    };

You can now easily use this function to retrieve large amounts of documents like this:

queryDocumentList('YOUR COLLECTION LINK', {query: 'SELECT * from c'})
.then(documents =>{
    console.log(documents)
})
.catch(err=>{
    console.error(err)
})

Hopefully this will make your search for a quick help into this topic easier, if you have any questions please comment bellow.

Vincent

Vincent van Wingerden

Vincent van Wingerden

I work for Microsoft as Technology advisor on the area of big data and advanced analytics. When I have made another cool demo I hope to write about it here. All opinions are my own

Read More