Fundamental concepts

Fauna is a serverless global database designed for low latency and developer productivity. FQL, its query language, was also designed with these goals in mind. With it, you can create expressive queries that allow you to harness the full power of Fauna.

In this five-part tutorial, we cover the basics of FQL with no prior knowledge necessary. If you are skimming and don’t understand something, you probably only need to go back to a previous page.

This first part of the tutorial begins a journey through space by looking at FQL and fundamental concepts of Fauna.

Getting started

Before embarking on our space adventure, you only need to signup for a free account: https://dashboard.fauna.com/accounts/register

The Fauna Dashboard signup screen

Once you have signed up and logged in, create a new database and you’re ready to get started:

The Dashboard New Database screen

It’s also possible to install Fauna on your development machine using an official Docker image if you prefer.

About documents and collections

Fauna is a document-oriented database. Instead of organizing data in tables and rows, it uses documents and collections.

The smallest units of data in Fauna are schemaless documents which are basically JSON with some extra types. These documents are grouped into collections which are simply buckets of documents.

This is what a simple document looks like:

{
  "ref": Ref(Collection("Planets"), "264471980339626516"),
  "ts": 1588478985090000,
  "data": {
    "name": "Vulcan"
  }
}

  • ref is a reference that uniquely identifies the document inside the Planets collection with the document ID 264471980339626516. We’ll go over references and the special Reference type in more detail later.

  • ts is a timestamp (expressed as a Long) of the document’s last event (e.g., create, read, update, delete) in microseconds.

  • data is the actual data of the document. You can create any structure you need and use any of the JSON and Fauna types, including Strings, Numbers, References to other documents, nested objects, Arrays, etc.

See Limits for document size, query size, and transaction time limits.

Your first collections

Obviously, before we begin our space adventure, we need a spaceship and a pilot. How else are we going to travel through space?

Let’s create a Spaceships collection using the CreateCollection function:

CreateCollection({name: "Spaceships"})
{
  "ref": Collection("Spaceships"),
  "ts": 1590269343560000,
  "history_days": 30,
  "name": "Spaceships"
}

As you can see, the result looks very similar to a document. All data in Fauna is stored in documents. For now, let’s leave the default values and move on.

Let’s create another a collection for our pilots:

CreateCollection({name: "Pilots"})

We’re ready now to start creating our first documents.

Basic CRUD operations

Create

Let’s create our first document with the Create function:

Create(
  Collection("Pilots"),
  {
    data: {
      name: "Flash Gordon"
    }
  }
)
{
  "ref": Ref(Collection("Pilots"), "266350546751848978"),
  "ts": 1590270525630000,
  "data": {
    "name": "Flash Gordon"
  }
}

The document you just created has an auto-generated document ID, which will be different from the one shown in the response above. Be sure to take note of your document ID and use it in subsequent examples.

Let’s break this down:

  • Create is used to create new documents.

  • Collection("Pilots") is a reference to the Pilots collection.

  • {data: {name: "Flash Gordon"}} is the actual data of the document.

So now that we have created a pilot, we can create a new spaceship:

Create(
  Collection("Spaceships"),
  {
    data: {
      name: "Millennium Hawk",
      pilot: Ref(Collection("Pilots"), "266350546751848978")
      // NOTE: be sure to use your pilot's document ID here
    }
  }
)

As you can see, we’re now storing a reference to another document in the pilot property. I will cover references and relationships in much more detail in part 3 of the tutorial.

SQL users might be tempted to store the document ID in a pilot_id property of the JSON instead of a Reference. This would be totally acceptable, but it’s recommended to use native references. Doing so makes your FQL queries much simpler as we’ll see later on.

Read

To read documents, we use the Get function which receives a document reference and returns an actual document:

Get(Ref(Collection("Spaceships"), "266354515987399186"))
  // NOTE: be sure to use your spaceship's document ID here
{
  "ref": Ref(Collection("Spaceships"), "266354515987399186"),
  "ts": 1590274311000000,
  "data": {
    "name": "Millennium Hawk",
    "pilot": Ref(Collection("Pilots"), "266350546751848978")
  }
}

Update

To update a document, we use Update. If we wanted to change the name of our ship, we’d simply run:

Update(
  Ref(Collection("Spaceships"), "266354515987399186"),
  // NOTE: be sure to use your ship's document ID here
  {
    data: {
      name: "Millennium Falcon"
    }
  }
)
{
  "ref": Ref(Collection("Spaceships"), "266354515987399186"),
  "ts": 1590274726650000,
  "data": {
    "name": "Millennium Falcon",
    "pilot": Ref(Collection("Pilots"), "266350546751848978")
  }
}

As you can see, only the name has been updated in the document and the pilot remains untouched. It’s also possible to replace an entire document using Replace instead.

Delete

On second thought, it’s probably better if we don’t use that copyrighted name for our spaceship. We don’t want to get into trouble with the galactic empire.

As expected, to delete a document we simply use Delete:

Delete(Ref(Collection("Spaceships"), "266354515987399186"))
{
  "ref": Ref(Collection("Spaceships"), "266354515987399186"),
  "ts": 1590274726650000,
  "data": {
    "name": "Millennium Falcon",
    "pilot": Ref(Collection("Pilots"), "266350546751848978")
  }
}

Let’s create a new spaceship again to continue with our adventure:

Create(
  Collection("Spaceships"),
  {
    data: {
      name: "Voyager",
      pilot: Ref(Collection("Pilots"), "266350546751848978")
    }
  }
)

Your first index

Fetching all documents in a database to check if each document fits a particular criteria would be very slow. In the relational world, this would be comparable in concept to a full table scan.

To solve this problem, Fauna implements indexes. These are database entities that organize your data in such a way that they allow for efficient lookup of multiple documents. Whenever you create new documents, the indexes that cover those documents are automatically updated.

In the next part of the tutorial, we show that indexes can span multiple collections and accept parameters for sorting and filtering.

For now, let’s create a simple index to list all the documents in a collection:

CreateIndex({
  name: "all_Pilots",
  source: Collection("Pilots")
})
{
  "ref": Index("all_Pilots"),
  "ts": 1590278778420000,
  "active": true,
  "serialized": true,
  "name": "all_Pilots",
  "source": Collection("Pilots"),
  "partitions": 8
}

Again, you can see that an index is just another type of document.

After adding some more pilots to our collection, we can query our new index like this:

Paginate(Match(Index("all_Pilots")))
{
  "data": [
    Ref(Collection("Pilots"), "266350546751848978"),
    Ref(Collection("Pilots"), "266359364060709394"),
    Ref(Collection("Pilots"), "266359371696439826"),
    Ref(Collection("Pilots"), "266359447111074322")
  ]
}

Let’s break this down:

  • Index returns a reference to an index.

  • Match accepts the index reference and returns a set, which is sort of like an abstract representation of the documents covered by the index. At this point, no data has been fetched yet.

  • Paginate takes the set returned by Match, reads the matching index entries, and returns a Page of results. In this case, this is simply an Array of References.

Using the Documents function to get all the documents of a collection

The previous index was actually a very simplistic example that served as an introduction to indexes.

Since retrieving all the documents in a collection is a very common need, you can use the Documents function to avoid the need to create a new index for every collection. It produces exactly the same results as the equivalent index.

Paginate(Documents(Collection('Pilots')))
{
  "data": [
    Ref(Collection("Pilots"), "266350546751848978"),
    Ref(Collection("Pilots"), "266359364060709394"),
    Ref(Collection("Pilots"), "266359371696439826"),
    Ref(Collection("Pilots"), "266359447111074322")
  ]
}

Page size

By default, Paginate returns pages of 64 items. You can define how many items you’d like to receive with the size parameter, up to 100,000 items:

Paginate(
  Match(Index("all_Pilots")),
  {size: 2}
)
{
  "after": [
    Ref(Collection("Pilots"), "266359371696439826")
  ],
  "data": [
    Ref(Collection("Pilots"), "266350546751848978"),
    Ref(Collection("Pilots"), "266359364060709394")
  ]
}

Since the number of results, in this case, does not fit in one page, Paginate also returns the after property to be used as a cursor.

Using Lambda to retrieve a list of documents

In some cases, you might want to retrieve a list of references, but generally, you will probably need an actual list of documents.

Initially, you might think the best way to solve this would be by performing multiple queries from your programming language. That would be an anti-pattern which you absolutely want to avoid. You would introduce unnecessary latency and make your application much slower than it needs to be.

For example, in this JavaScript example, you’d be waiting first for the query to get the references and then for the queries to get the documents:

// Don't do this!
const result = await client.query(q.Paginate(q.Match(q.Index("all_Pilots")));
const refs = result.data;
const promises = result.map(refs.map(ref => client.query(q.Get(ref))));
const pilots = await Promise.all(promises);

Or even worse, by waiting for each and every query that gets a document:

// Don't do this!
const result = await client.query(q.Paginate(q.Match(q.Index("all_Pilots")));
const refs = result.data;
const pilots = [];
for (const ref of refs) {
  const pilot = await client.query(q.Get(ref));
  pilots.push(pilot);
}

The solution is simply to use FQL to solve this neatly in a single query.

Here’s the idiomatic solution of getting an actual list of documents from an array of references:

Map(
  Paginate(Match(Index("all_Pilots"))),
  Lambda('pilotRef', Get(Var('pilotRef')))
)
{
  "data": [
    {
      "ref": Ref(Collection("Pilots"), "266350546751848978"),
      "ts": 1590270525630000,
      "data": {
        "name": "Flash Gordon"
      }
    },
    {
      "ref": Ref(Collection("Pilots"), "266359364060709394"),
      "ts": 1590278934520000,
      "data": {
        "name": "Luke Skywalker"
      }
    },
    // etc...
  ]
}

We’ve already seen that Paginate returns an array of references, right? The only mystery here is the use of Map and this Lambda thing.

You’ve probably already used a "map" function in your programming language of choice. It’s a function that accepts an array and returns a new array after performing an action on each item.

Consider this JavaScript example:

const anotherArray = myArray.map(item => doSomething(item));

// which is equivalent to:

const anotherArray = myArray.map(function (item) {
  return doSomething(item);
});

With this in mind, let’s break down this portion of our FQL query:

Map(
  Paginate(Match(Index("all_Pilots"))),
  Lambda("pilotRef", Get(Var("pilotRef")))
)

  • Paginate returns an array of references.

  • Map accepts an array (from Paginate or other sources), performs an action on each item of this array, and returns a new array with the new items. In this case, the action is performed using Lambda, which is the Fauna equivalent of what you’d call a simple anonymous function in JavaScript. It’s all very similar to the previous JavaScript example.

  • Lambda 'pilotRef' defines a parameter called pilotRef for the anonymous function. You can name this parameter anything that makes sense for you. In this example, the parameter receives a reference, which is why I named it pilotRef.

  • Var is used to evaluate variables. In this case, it evaluates the variable named pilotRef and returns the document reference.

  • Get receives the reference and returns the actual document.

If we were to rewrite the previous FQL query with the JavaScript driver, we could do something like this:

q.Map(
   q.Paginate(q.Match(q.Index("all_Pilots"))),
  (pilotRef) => q.Get(pilotRef)
)

// Or:

q.Map(
   q.Paginate(q.Match(q.Index("all_Pilots"))),
   q.Lambda("pilotRef", q.Get(q.Var("pilotRef")))
)

You can paste JavaScript queries into the Web Shell as well as FQL queries.

The Dashboard shell demonstrating a JavaScript query

Using Let and Select to return custom results

Up until now, our documents have been pretty simple. Let’s add some more data to our spaceship:

Update(
  Ref(Collection("Spaceships"),"266356873589948946"),
  {
    data: {
      type: "Rocket",
      fuelType: "Plasma",
      actualFuelTons: 7,
      maxFuelTons: 10,
      maxCargoTons: 25,
      maxPassengers: 5,
      maxRangeLightyears: 10,
      position: {
        x: 2234,
        y: 3453,
        z: 9805
      }
    }
  }
)
{
  "ref": Ref(Collection("Spaceships"), "266356873589948946"),
  "ts": 1590524958830000,
  "data": {
    "name": "Voyager",
    "pilot": Ref(Collection("Pilots"), "266350546751848978"),
    "type": "Rocket",
    "fuelType": "Plasma",
    "actualFuelTons": 7,
    "maxFuelTons": 10,
    "maxCargoTons": 25,
    "maxPassengers": 5,
    "maxRangeLightyears": 10,
    "position": {
      "x": 2234,
      "y": 3453,
      "z": 9805
    }
  }
}

Cool.

So now imagine our application were in fact managing a whole fleet and you needed to show a list of ships to the fleet admiral.

First, we’d need to create an index:

CreateIndex({
  name: "all_Spaceships",
  source: Collection("Spaceships")
})

Ok, now we just use Paginate, Map, and Lambda (like we saw earlier) to get all of the documents. So we do that but…​ Oh no!

The fleet admiral is very unhappy about the slow performance of his holo-map now.

Sending the complete list with thousands of documents across light years of space wasn’t a great idea because it’s a lot of data. We propose breaking down the results with pages, but the admiral absolutely needs to see all ships at once.

"By the cosmic gods! I don’t care how much fuel a ship has!" shouts the admiral. "I only want to know its name, id, and position!".

Of course! Let’s do that:

Map(
  Paginate(Match(Index("all_Spaceships"))),
  Lambda("shipRef",
    Let(
      {
        shipDoc: Get(Var("shipRef"))
      },
      {
        id: Select(["ref", "id"], Var("shipDoc")),
        name: Select(["data", "name"], Var("shipDoc")),
        position: Select(["data", "position"], Var("shipDoc"))
      }
    )
  )
)
{
  "data": [
    {
      "id": "266356873589948946",
      "name": "Voyager",
      "position": {
        "x": 2234,
        "y": 3453,
        "z": 9805
      }
    },
    {
      "id": "266619264914424339",
      "name": "Explorer IV",
      "position": {
        "x": 1134,
        "y": 9453,
        "z": 3205
      }
    }
    // etc...
  ]
}

Boom! Now the holo-map loads much faster. We can see the satisfaction in the admiral’s smile.

Since we already know how Paginate, Map, and Lambda work together, this is the new portion:

Let(
  {
    shipDoc: Get(Var("shipRef"))
  },
  {
    id: Select(["ref", "id"], Var("shipDoc")),
    name: Select(["data", "name"], Var("shipDoc")),
    position: Select(["data", "position"], Var("shipDoc"))
  }
)

Let

Let is a function used in FQL to create custom objects. You can even have nested Let functions to format the data with total freedom.

The first parameter of Let is used to define variables that are going to be used later on. These are called "bindings". These bindings are available to any nested Let objects that you create.

Here we define a shipDoc variable which stores the document returned from Get, which in turn uses the reference from the Lambda parameter:

{
  shipDoc: Get(Var("shipRef"))
}

The second portion is the actual object that defines all of the variables and their values:

{
  id: Select(["ref", "id"], Var("shipDoc")),
  name: Select(["data", "name"], Var("shipDoc")),
  position: Select(["data", "position"], Var("shipDoc"))
}

Select

The Select function is used to select data from objects or arrays.

Select(["data", "name"], Var("shipDoc"))

Here, we’re selecting the name property from the data property of the document stored in the shipDoc binding.

This array-like notation ["data", "name"] is called a "path". We’re using it here to get to the name property, but it can be used with integers to access array items too.

Conclusion

So that’s it for today. Hopefully, you learned something valuable!

In part 2 of the tutorial, we continue our space adventure by going deeper into indexes.

Is this article helpful? 

Tell Fauna how the article can be improved:
 Visit Fauna's forums or email docs@fauna.com

Thank you for your feedback!