15 September 2023 in back-end

MongoDB M001 course notes

nosql
MongoDB
notes

Contents

Intro

This post is just a bunch of notes that I took during the MongoDB M100 introductory course in October 2021. It wasn't put together as a blog post but you might still find it useful.

A NoSQL Document Database

ie. the data is stored in documents and the documents are stored in collections

Document = A way to organize and store data as a set of field-value pairs. Similar to JSON or other object structures

Collection = an organized store of documents in MongoDB, usually with common fields between documents.

There can be many collections per database and many documents per collection.

Atlas

Background

  • A fully managed cloud database service
  • Where Atlas is used, a MongDB is deployed to a cluster
  • cluster = group of servers that store your data
  • Clusters contain replica sets
  • replica sets are duplicated database structures that protect from loss
  • Has a free tier
  • 512Mb of data
  • 3 servers in a cluster
  • To access: https://cloud.mongodb.com

Getting started steps:

1 Create an organisation

  • Click username (top right)
  • In drop-down, select Organizations
  • Click Create New Organization
  • What is an organisation?
    • It is essentially a group of projects and team members. It is at this level that billing takes place. The cost of projects within an organisation are billed at the organisation level

2 Name organisation

  • Complete name field
  • select MogoDB Atlas in Features section

3 Click Create Organisation

4 Click New Project

  • What is a project?
    • Well it's exactly what you would expect. A self-contained piece of work
    • Access to projects can be grouped by team to make it simpler to control who has access to what project
    • Each project can have multiple Atlas clusters

5 Enter a name for the project in input field and hit Next

6 Click Create Project to ... well ... create the project!

7 On Database Deployments page click Build a Database

8 Select Free & Hobby and hit Create

9 Select closest geographical location and change Cluster Name (at bottom) to Sandbox then click Create Cluster

  • Takes a few minutes to deploy the Atlas Cluster

10 Select Connect view

A dialog opens to:

  • Add a connection IP address - select Allow Access From Anywhere for now NOTE: this is bad practice normally I even got an email warning me that I had made this selection!

11 Create a Database User

  • username: m001-student

  • password: m001-mongodb-basics

  • Click Create Database User

  • Close the Connect to Sandbox dialog (don't click Choose a connection method)

  • Select the ... tab and choose Load Sample Dataset then confirm choice

Connecting to the newly created Cluster via in-browser IDE

  • Click Connect
  • Connect with MongoDB Shell
  • In the Connect to Sandbox dialog:
    • Click I have the MongoDB Shell installed (it's part of the in-browser IDE that comes with th ecourse.)
  • Select the appropriate mongo shell version
  • Copy the connection script and paste into Terminal tab of IDE

Documents

  • Contain JSON
  • Stored as BSON (Binary JSON)

    • Provides some additional functionality beyond that of JSON

      • Dates
      • Binary data
      • Specific number types (int, float, long, decimal)

    • BSON can broadly be thought of as JSON but you need to be aware of potential language differences with certain types of data - check your language's MongoDB Driver for specs

    • https://www.mongodb.com/json-and-bson

    • https://bsonspec.org/

Importing & exporting data to/from MongoDB

  • Can import and export in both JSON & BSON

  • Use JSON if it needs to be human-readable

  • JSON

    • import command mongoimport
    • export command mongoexport

  • BSON

    • import command mongorestore
    • export command mongodump

Example shell commands:

mongodump --uri "mongodb+srv://<your username>:<your password>@<your cluster>.mongodb.net/sample_supplies"

mongoexport --uri="mongodb+srv://<your username>:<your password>@<your cluster>.mongodb.net/sample_supplies" --collection=sales --out=sales.json

mongorestore --uri "mongodb+srv://<your username>:<your password>@<your cluster>.mongodb.net/sample_supplies"  --drop dump

mongoimport --uri="mongodb+srv://<your username>:<your password>@<your cluster>.mongodb.net/sample_supplies" --drop sales.json

Note the --drop options to prevent duplicate entry errors on import

Accessing data

Using the Atlas Data Explorer

Click the Collections tab in the project Databases screen

The left-hand panel shows a list of databases that can be expanded to show their collections

Clicking a collection opens it in the main window

Filtering the collection

The Filter box is at the top just below the horizontal nav section

Filter queries must be entered in valid JSON

  • eg: {"state": "NY"} or, more targetted,
  • {"state": "NY", "city": "ALBANY"}

Using the MongoDB Shell

  1. Connect to the Atlas cluster:
mongo "mongodb+srv://sandbox.uqywj.mongodb.net/myFirstDatabase" --username m001-student

Some example commands:

show dbs #lists databases on the cluster (including admin & local)

use sample_training #selects the sample_training database

show collections #views the collections in the selected database

db.zips.find({"state": "NY"})
# db points to the sample_training database 'use'd earlier
# filter criteria are the same as we put in the filter field in the UI above
# command returns the first 20 documents

it #iterates through the collection showing 20 at a time

db.zips.find({"state": "NY"}).count() #returns the number of documents matching the query

db.zips.find({"state": "NY", "city": "ALBANY"}) #reurns the documents matching the narrower query criteria

db.zips.find({"state": "NY", "city": "ALBANY"}).pretty() #same but with each document more readable

Inserting data

Using the Atlas Data Explorer

  • Can insert field-by-field or as a full JSON document
  • Data types are specified for each key:value pair (in field-by-field view)
  • every document must have an uniquekey & value
    • That is the _id key:value
    • the _id value, if auto-generated, is of type ObjectId

Note: a document doesn't have to have a _id value of type ObjectId if there is another field that can serve as a truly unique identifier but in most cases using an ObjectId _id is the best plan.

  • If _id isn't provided in the insert() command, an ObjectId will be automatically generated.

Using the MongoDB Shell

Some example commands:

mongo "mongodb+srv://sandbox.uqywj.mongodb.net/myFirstDatabase" --username m001-student

use sample_training #selects the sample_training database

db.inspections.findOne(); #selects a single random document

db.inspections.insert({
      "_id" : ObjectId("56d61033a378eccde8a8354f"),
      "id" : "10021-2015-ENFO",
      "certificate_number" : 9278806,
      "business_name" : "ATLIXCO DELI GROCERY INC.",
      "date" : "Feb 20 2015",
      "result" : "No Violation Issued",
      "sector" : "Cigarette Retail Dealer - 127",
      "address" : {
              "city" : "RIDGEWOOD",
              "zip" : 11385,
              "street" : "MENAHAN ST",
              "number" : 1712
         }
  })

#Fails because _id already exists

db.inspections.insert({
      "id" : "10021-2015-ENFO",
      "certificate_number" : 9278806,
      "business_name" : "ATLIXCO DELI GROCERY INC.",
      "date" : "Feb 20 2015",
      "result" : "No Violation Issued",
      "sector" : "Cigarette Retail Dealer - 127",
      "address" : {
              "city" : "RIDGEWOOD",
              "zip" : 11385,
              "street" : "MENAHAN ST",
              "number" : 1712
         }
  })

#doesn't fail because a new _id is auto-generated.
#you just end up with a duplicated document

db.inspections.find({"id" : "10021-2015-ENFO", "certificate_number" : 9278806}).pretty()

#returns the two duplicated documents

Inserting multiple documents

To insert multiple documents, simply structure them as an array of JSON objects and pass to the .insert() method

db.inspections.insert([ { "test": 1 }, { "test": 2 }, { "test": 3 } ])

The default behaviour is to insert the documents in the order that they appear in the array.

If a duplicate _id value is encountered, the process will simply cease at that point:

db.inspections.insert([{ "_id": 1, "test": 1 },{ "_id": 1, "test": 2 },{ "_id": 3, "test": 3 }])

In this example, test 1 is inserted but the process ends when a duplicate _id is encountered with test 2. Test 3 is never even considered.

If { "ordered": false } is added as an argument after the array, the order is ignored and all non-duplicate-_id documents are inserted

db.inspections.insert([{ "_id": 1, "test": 1 },{ "_id": 1, "test": 2 },{ "_id": 3, "test": 3 }], { "ordered": false })

In this example tests 1 and 3 are inserted (although, if this is run after the statement above, test 1 will not be inserted because it already exists with the same _id value.)

Updating documents

Using the Atlas Data Explorer

with a collection of documents shown in the main project screen

  • when hovering over a document, edit, copy, clone & delete icons are shown over on the right-hand side
  • click edit and the document becomes editable
  • remember the data types need to be set
  • Also remember that the document can have sub-structures nested in iit like:
    • other objects
    • arrays
    • arrays of objects etc

Using the MongoDB Shell

Connect to the Atlas Cluster (same command as shown above)

There are two update commands:

  • updateOne()
  • updateMany()

Examples

Using MQL update operators

use sample_training
db.zips.find({ "city": "HUDSON" }).count()
# this reveals there are 16 cities so named

# we want to increase the population (`pop`) of each by 10
db.zips.updateMany({ "city": "HUDSON" }, { "$inc": { "pop": 10 } })

# next we want to set the population at zip 12543
db.zips.updateOne({ "zip": "12534" }, { "$set": { "pop": 17630 } })

Note if the field that you try to update doesn't exist then MQL will simply assume that you intended to create a new field in that document

# if we set field `population` when we mean `pop` we will end up with a
# pop field and a population field
db.zips.updateOne({ "zip": "12534" }, { "$set": { "population": 17630 } })

Updating an Array field using &push

db.grades.updateOne({ "student_id": 250, "class_id": 339 },
                    { "$push": { "scores": { "type": "extra credit",
                                             "score": 100 }
                                }
                     })

Find out more about MQL update operators here: https://docs.mongodb.com/manual/reference/operator/update/#id1

Deleting data

Using Atlas Data Explorer

Simply click the dustbin icon alongside the document, collection or database

Using MongoDB Shell

After connecting to the Atlas Cluster we can delete documents with:

  • deleteOne()
  • delteMany()

Examples:

use sample_training
# delete all documents with "test" field of 1
db.inspections.deleteMany({ "test": 1 })

# delete one document with "test" field of 3
# NOTE: this will just delete the first one found that matches "test": 3
# any others that match will be left alone
db.inspections.deleteOne({ "test": 3 })

Consider this last example. It will almost always be the case that deleteOne() is used exclusively where the match criteria uses _id so you can be confident that there is only one matching document in the collection.

Deleting a collection

db.inspection.drop()
# this will delete the collection called 'inspection' from the currently selected Db

Comparison Operators (equals, less than etc etc)

Syntax:

{ <field>: { <operator>: <value> } }

eg. to return all trips <= 70 seconds:

{ "tripduration": { "$lte": 70 } }

Available operators

  • $eq included implicitly. see example below
{ "tripduration": { "$eq": 70 } }
// is the same as 
{ "tripduration": 70 }
  • $ne
  • $lt
  • $gt
  • $lte
  • $gte

Using Atlas Data Explorer

Simply include the required comparison operator in the filter box.

So, to return all trips <= 70 seconds just enter the string precisely as shown above in the filter box

Using MongoDB Shell

Using the find() command, include the comparison operator(s) in your query

eg.

db.trips.find({ "tripduration": { "$lte": 70 } })

so exactly the same string, just wrapped in the find() command

Logic Operators

Available operators

  • $and included implicitly where multiple criteria. see examples below
{ "$and": [{sector: "Mobile Food Vendor - 881"}, {result: "Warning"}] }
// is the same as
{ sector: "Mobile Food Vendor - 881", result: "Warning" }

{ "$and": [{"student_id": {"$gt": 25}}, {"student_id": {"$lt": 100}}] }
// is the same as
{ "student_id": {"$gt": 25, "$lt": 100} }
  • $or
  • $nor
  • $not

Syntax:

$and, $or, $nor

{ <operator>: [{statement1}, {statement2}, ...] }

$not

{ $not: {statement} }

Since $and is implicitly included where there are multiple criteria, when do we explicitly use the $and operator?

  • Answer - when the same operator needs to be used more than once in the same query.

Example:

db.routes.find({ "$and": [ { "$or" :[ { "dst_airport": "KZN" },
                                    { "src_airport": "KZN" }
                                  ] },
                          { "$or" :[ { "airplane": "CR2" },
                                     { "airplane": "A81" } ] }
                         ]}).pretty()

One of the test labs was quite challenging!

Q: To complete this exercise connect to your Atlas cluster using the in-browser IDE space at the end of this chapter.

How many companies in the sample_training.companies dataset were

either founded in 2004

[and] either have the social category_code [or] web category_code,

[or] were founded in the month of October

[and] also either have the social category_code [or] web category_code?

Solution: (answer was 149)

db.companies.find(
  {"$and":
	[
      {"$or":
       [
         {category_code: "social"},
         {category_code: "web"}
       ]
      },
      {"$or":
       [
         {founded_year: 2004},
         {founded_month: 10}
       ]
      }
    ]
  }
).count()

Expressive Query Operator

This includes another use of the $

We saw already that it is used to denote comparison operators and logic oiperators

It is used in Expressive Operators to denote the value of a field in a given document

so "$start station id" refers to the value of start station id in the document under considration.

Some simple examples:

  • Finding the number of trips where the start station ans end station are the same:
db.trips.find(
  { "$expr": 
   { 
    "$eq": [ 
      "$end station id", 
      "$start station id"
    ]
   }
  }
).count()
  • Adding another parameter - finding the number of trips where the start station and end station are the same and the duration is > 10 mins:
db.trips.find(
  { "$expr":
   { 
    "$and": [ 
      { "$gt": [ "$tripduration", 1200 ]},
      { "$eq": [ "$end station id", "$start station id" ]}
    ]
   }
  }
).count()

Note the syntax in these examples

In comparison operators syntax is { <field>: { <operator>: <value> } }

In expressive query operators the sytax is { <operator>: { <field>: <value> } }

This is Aggregation pipeline syntax. We will learn more about the Aggregation Pipeline in later Chapters

Array Operators

MongoDB Manual: https://docs.mongodb.com/manual/tutorial/query-arrays/

Already seen $push

  • adds an element to an array (it's JavaScript after all)
  • also adds an array field into a document where the field didn't exist.

Note: if the field already exists and it is a different type it will error.

Exploring other Array Operators using the sample_airbnb.listingsAndReviews collection

  • $all will return all documents that contain at least the elements listed in the query in the array field also listed in the query

  • $size will return documents that list that number of elements in the selected array field

Here's a quick example:

db.listingsAndReviews.find({ "amenities": {
                                  "$size": 20,
                                  "$all": [ "Internet", "Wifi",  "Kitchen",
                                           "Heating", "Family/kid friendly",
                                           "Washer", "Dryer", "Essentials",
                                           "Shampoo", "Hangers",
                                           "Hair dryer", "Iron",
                                           "Laptop friendly workspace" ]
                                         }
                            }).pretty()

This query will find all documents with exactly 20 amenities which include all the amenities listed in the query array

You can limit the length of returned documents adding a projection to the find query. More on projections later.

Example from Lab 2:

Q:

Using the sample_airbnb.listingsAndReviews collection find out how many documents have the "property_type" "House", and include "Changing table" as one of the "amenities"?

A:

db.listingsAndReviews.find(
  {
    "$and": 
    [
      {"property_type": "House"},
      {"amenities": 
       {
         "$all":
         ["Changing table"]
       }
      }
    ]
  }
).count()

Array Operators & Projection

MongoDB documents can contain a lot of fields. If you don't want to include all of the fields in the response you can make use of Projection to limit the fields.

Projection

  • You can choose to specify the fields you do wantwith a 1

-OR-

  • specify the fields you don't want with 0

Example: Find all documents with exactly 20 amenities which include all the amenities listed in the query array, and display their price and address:

db.listingsAndReviews.find({ "amenities":
        { "$size": 20, "$all": [ "Internet", "Wifi",  "Kitchen", "Heating",
                                 "Family/kid friendly", "Washer", "Dryer",
                                 "Essentials", "Shampoo", "Hangers",
                                 "Hair dryer", "Iron",
                                 "Laptop friendly workspace" ] } },
                            {"price": 1, "address": 1}).pretty()

The _id field is returned by default. If you don't want that field you need to explicitly add "_id": 0. That is the only time you can use 1s and 0s in the same command.

Example: Find all documents that have Wifi as one of the amenities only include price and address in the resulting cursor:

db.listingsAndReviews.find({ "amenities": "Wifi" },
                           { "price": 1, "address": 1, "_id": 0 }).pretty()

$elemMatch

Syntax

{ <field>: { "elemMatch": { <field>: <value> }}}

Can be used to:

  • Match documents that contain an array field with at least one element that matches the specified query criteria

-OR-

  • Added to a projection to return only the array elements with at least one element that matches the specified criteria

Examples:

  • Find all documents where the student in class 431 received a grade higher than 85 for any type of assignment:
db.grades.find({ "class_id": 431 },
               { "scores": { "$elemMatch": { "score": { "$gt": 85 } } }
             }).pretty()
  • Find all documents where the student had an extra credit score:
db.grades.find({ "scores": { "$elemMatch": { "type": "extra credit" } }
               }).pretty()
  • How many companies in the sample_training.companies collection have offices in the city of Seattle?
db.companies.find({"offices": {"$elemMatch": { "city": "Seattle"}}}).count()

Array Operators & Sub-documents (dot notation)

Dot notation

To see dot notation in action, here is a document from the trips collection of the sample_training Db:

_id: 572bb8222b288919b68abf5a
tripduration: 379
start station id: 476
start station name: "E 31 St & 3 Ave"
end station id: 498
end station name: "Broadway & W 32 St"
bikeid: 17827
usertype: "Subscriber"
birth year: 1969
start station location: Object
  type: "Point"
  coordinates: Array
    0: -73.97966069
    1: 40.74394314
end station location: Object
  type: "Point"
  coordinates: Array
    0 :-73.98808416
    1 :40.74854862
start time: 2016-01-01T00:00:45.000+00:00
stop time: 2016-01-01T00:07:04.000+00:00

Notice that the start station location and end station location fields contains documents with type and coordinates fields. And the coordinates field contains an array.

So to target documents with start station location of type Point our query would be:

db.trips.find({"start station location.type": "Point"})

Note the way that the whole path to the nested document sits within quotes

Another example: to find companies where the first person in the relationships array has the last name "Zuckerberg"...

db.companies.find({ "relationships.0.person.last_name": "Zuckerberg" },
                  { "name": 1 }).pretty()

Notice the Projection { "name": 1 } which limits results to just the name field (plus _id because it isn't explicitly excluded)

Matching with RegEx

MongoDB manual on $regex: https://docs.mongodb.com/manual/reference/operator/query/regex/

Aggregation Framework (ie. Filtering & Grouping the Stream of data)

Simple terms - the aggregation framework is another way to query MongoDB

Everything that can be done in MQL (Mongo Query Language) can be done in Aggregation Framework

It works in a similar way to Streams in Java or chaining Array iterators in JavaScript

Syntax

Example query: Find all documents that have Wifi as one of the amenities. Only include price and address in the resulting cursor.

In MQL:

db.listingsAndReviews.find({ "amenities": "Wifi" },
                           { "price": 1, "address": 1, "_id": 0 }).pretty()

In aggregate framework:

db.listingsAndReviews.aggregate([
                                  { "$match": { "amenities": "Wifi" } },
                                  { "$project": { "price": 1,
                                                  "address": 1,
                                                  "_id": 0 }}]).pretty()

Notes on this example:

.aggregate

  • [] - array notation - the order of actions is important. Each action changes the shape of the data that is passed to the next action in the array

  • "$match" stage - effectively a filter. Only documents matching the criteria pass through to the next action

  • "$project" stage - this is the projection stage. The criteria set here determine which fields will be included in the result set from whatever documents we have left at the start of this stage.

Why Aggregation Framework?

It is a very flexible way to work with data.

Much more flexible than MQL

The GROUPING stage

In this stage the data at the start of the stage is filtered in multiple groups (or reservoirs as the lecture suggested)

Syntax fundamentals:

{ $group:
	{
      _id: <field>, // the field by which the data is to be grouped
      <field>: { <accumulator1>: <expression1>},
      ...
    }
}

Examples using airbnb.listingsAndReviews collection:

  1. Project only the address field value for each document, then group all documents into one document per address.country value.
db.listingsAndReviews.aggregate([ { "$project": { "address": 1, "_id": 0 }},
                                  { "$group": { "_id": "$address.country" }}])

  • This finds all documents, projects just the address field to the cursor and then groups that data by address.country

  • NOTE that the field name must have a $ prefix

  1. Project only the address field value for each document, then group all documents into one document per address.country value, and count one for each document in each group.
db.listingsAndReviews.aggregate([
                                  { "$project": { "address": 1, "_id": 0 }},
                                  { "$group": { "_id": "$address.country",
                                                "count": { "$sum": 1 } } }
                                ])
  • This does the same as the previous example and then performs a count (eqivalent of a reduce() operation) and adds 1 to $sum for each occurence of each country.

Learn more about the Aggregation Framework in: https://university.mongodb.com/courses/M121/about

sort(), limit() & skip()

  • sort & limit are common DB commands

sort()

  • Ascending order of "pop"
db.zips.find().sort({ "pop": 1 })
  • Descending order of "pop"
db.zips.find().sort({ "pop": -1 })
  • Ascending order of "pop" then descending order of "city"
db.zips.find().sort({ "pop": 1. "city": -1 })

limit()

  • Ascending order of "pop" - first 10 documents
db.zips.find().sort({ "pop": 1 }).limit(10)

NOTE: Using limit() without (or before) sort() makes little sense. So MongoDB assumes when you write .limit().sort() you actually mean .sort().limit()

skip()

Tells MongoDB how many documents to skip before it starts to return results

Commonly used in Pagination

Full info here: https://docs.mongodb.com/manual/reference/method/cursor.skip/

Introduction to Indexes

Adding an index is the most impactful way to improve the efficiency of your queries

It is a data structure that stores a small portion of the collection's data in an easy to traverse form

  • ie a data structure that optimises queries

You should build indexes that support your queries.

For example, in the trips collection, let's say you find that you often compose queries based on the birth year. Then you should create an index for birth year

db.trips.createIndex({ "birth year": 1 })

This creates a single field index on that field in increasing order.

Having built that index, when MongoDB encounters a query like:

db.trips.find({"birth year": 1989})

it doesn't have to search through all documents to find those with birth year 1989, it can simply go to the index and extract the 1989 block (because the index is pre-sorted)

That index will even help in queries like this:

db.trips.find({ "start station id": 476 }).sort( { "birth year": 1 } )

because the sort() element has already been done in advance. This cuts out a lot of processing work.

You can also create compound indexes on multiple fields

eg.

db.trips.createIndex({ "start station id": 1, "birth year": 1 })

This creates an index sorted first on start station id then on birth year

This can co-exist in the trips collection with the documents and the single field index created earlier

Learning more

  • MongoDB Performance course

Data Modelling

"A way to organize fields in a document to support your application performance and querying capabilities."

MongoDB leaves it up to you to decide on the best model

Rule of thumb

Data should be stored in the way that it is used. In other words:

data that is used together should be stored together

This will determine the shape of documents and the number of collections. Data models should evolve over time. MongoDB makes that as seamless as possible.

Useful blog post: https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-1

Upsert

A command that can:

  • either update an existing document

  • or insert a new document

  • depending on certain criteria contained in the command

Remember - everything that we have learned about locating a document in MQL can also be used to modify that document

ie:

db.collection.updateOne({<query to locate>}, {<update>})

Upsert is a hybrid of update & insert. When used, it is the 3rd directive coming after <update>

db.collection.updateOne({<query to locate>}, {<update>}, {"upsert": true})

By default upsert is set to false. It should only be used when needed!

Example:

Imagine a smart home with a number of IOT sensors recording data continually.

Our database contains documents that hold arrays of readings for specific sensors with a maximum of 48 readings per document array.

When the array has 48 readings a new document is created.

A suitable command would be:

db.iot.updateOne(
  { "sensor": r.sensor, "date": r.date, "valcount": { "$lt": 48 } },
  { "$push": { "readings": { "v": r.value, "t": r.time } }, "$inc": { "valcount": 1, "total": r.value } },
  { "upsert": true })

Walking through this...

  • First: we have the <query to locate> a matching document

    • The reading will contain a variable that identifies the sensor r.sensor and date r.date
    • We want to check whether a document exists with those field:value entries
    • The matched document also needs a valcount of less than 48 (max array length)
    • If these criteria are all matched, we want to update the record that matches

  • Next, the <update> directive will:

    • $push a new object to the readings array containing the reading value (r.value) and the time of the reading (r.time)
    • and $inc the valcount value because a new element has been added to the array
    • (note: it is more efficient to maintain a valcount field rather than have to check array length every time)

  • Next, {"upsert": true} is telling Mongo:

    • "If we don't find a match in an existing document (in line with the first directive above) insert a new one.
    • When a new document is inserted, notice the following things...
    • _id is automatically added
    • MongoDB knows that readings needs to be an array field because of the $push command
    • the $inc command tells MongoDB to
      • initialise valcount with the incremental value 1
      • initialise total with r.value

As this example demonstrates, upsert can be very powerful. But we have to be mindful of its effects. In the event of an insert will the document be correctly structured when compared to other documents in the collection? You don't want documents with a random schema!

In many cases we are better to use the "vanilla" update (with upsert: false) or insert commands.

Atlas Data Explorer - advanced features

So far we have only used the Find tab of the data explorer.

Other tabs:

Indexes

This shows the indexes that have been created, how many times they have been accessed together with some performance criteria

Schema Anti-Patterns

This provides some data modelling advice once your data has been accessed enough times

Aggregation

This is a super-useful tool for building stages in your aggregation framework.

A dropdown list of options is provided for each element of the pipeline making setup that much easier

You can drag and drop pipeline stages to alter the order and immediately see the effect on the data passing through the pipeline.

And, when you are happy that the pipeline structure is right, you can export it in the language of your choice together with syntax for imports & drivers!

Search indexes

An advanced text search feature

There is a tutorial for that feature here: https://docs.atlas.mongodb.com/reference/atlas-search/tutorial/#fts-tutorial-ref

Atlas Realm The Developer solution (see M220 courses below)

Managed services to connect your data to mobile apps, websites and services.

At first view it appears that Realm can be a combination of backend app & database! Is that too good to be true??

Realm has its own documentation here: https://docs.mongodb.com/realm/

Atlas Charts

Built-in charting functionality from the MongoDB data on cloud.mongodb.com

You select the data that you want to visualise, the type of visualisation & MongoDB puts together the required aggregation framework in the background (this is visible from within the Charts tab) and creates the visualisation.

More info is available in the Charts tutorial: https://docs.mongodb.com/charts/saas/tutorial/order-data/order-data-tutorial-overview/

MongoDB Compass

A downloadable UI Client to connect to your Atlas clusters

Similar to Atlas Data Manager but with many more features and capabilities to navigate & analyse your data

Useful links:

MongoDB Developer Hub: https://www.mongodb.com/developer/

MongoDB Community Forums: https://www.mongodb.com/community/forums/

MongoDB for financial services: https://www.mongodb.com/industries/financial-services