Use All the Databases – Part 2

Published

Loren Sands-Ramshaw, author of GraphQL: The New REST shows how to combine data from multiple data sources using GraphQL in part two of this Write Stuff series.

In Part 1 I introduced the app we’re building, the databases we’re using, the what and why of GraphQL, how to write a GraphQL query and schema, and how to set up and run a GraphQL server. Now we’ll finish writing the server code by querying all of our data sources. If you'd like to follow along with the code, start on the empty-resolvers branch of the server repo:

git clone git@github.com:GraphQLGuide/all-the-databases.git  
git checkout empty-resolvers  

And then finish the setup instructions to run locally. If you run into problems, you can check your work against the master branch, which has the completed server code.

Part 1

Part 2

Resolvers

The last thing we need are resolvers, which take a field and resolve it into a value by looking it up from a data source and returning it.

SQL

Let's start with a user query:

User query

Here's how we format the resolver function for the user query:

const resolvers = {  
  Query: {
    user(_, args) {
      // args.id will be 1 in the above example
    },
  },
}

code

We need to return the user who has an id of 1, and our users are stored in Postgres, so we want to execute this SQL query:

SELECT * FROM `users` AS `user` WHERE (`user`.`id` = 1);  

which in the Sequelize ORM is done with User.find:

const resolvers = {  
  Query: {
    user(_, args) {
      return User.find({
        where: { id: args.id },
      });
    },
  },
}

code

Elasticsearch

Each field in a GraphQL query needs to be resolved—not just the top-level query name. In the above basic user query, the GraphQL server knew how to resolve the firstName, lastName, and photo fields because they were attributes of the user object returned from the user resolver. However, that object didn't have a mentions attribute, so if we want the below query to work, we'll need to add a resolver for mentions:

User with mentions

According to the schema, the user query returns an object of type User, and mentions is a field on a User. So we define the resolver accordingly:

const resolvers = {  
  Query: {
    user(_, args) { ... },
  },
  User: {
    mentions(user) {
      // fetch and return mentions
    },
  },
}

code

We get the user object as an argument (the one returned from the Query.user resolver), and we want to search Elasticsearch for all tweets that contain that user's name. Once we get the results from the database, we wrap them in our Sequelize Tweet model with Tweet.build so they're easier to work with:

User: {  
  async mentions(user) {
    const results = await Elasticsearch.search({ q: `${user.firstName} ${user.lastName}` });
    return results.hits.hits.map(hit => Tweet.build(hit._source));
  },
},

code

But we're not done yet! The docs we got from Elasticsearch look like this:

{
  "text": "Maurine Rau Eligendi in deserunt.",
  "userId": 1,
  "city": "San Francisco",
  "created": 1481742701457
}

The GraphQL server finds the text, city, and created fields, but doesn't know what to do for the author or views fields, so we need to write resolvers for those:

const resolvers = {  
  Query: {
    user(_, args) { ... },
  },
  User: {
    async mentions(user) { ... },
  },
  Tweet: {
    author(tweet) {
      // fetch and return author user doc
    },
    views(tweet) {
      // fetch and return the number of views
    },
  },
}

code

We know what to do in the author resolver—fetch the right user doc from Postgres. We do that with a SELECT statement on the users table where user.id is equal to tweet.userId (tweet we get as an argument to the resolver). If we set up our ORM right (with TweetModel.belongsTo(UserModel)), we can do that with just tweet.getUser():

Tweet: {  
  author(tweet) {
    return tweet.getUser();
  },
}

code

One cool thing about GraphQL is resolvers work at any query depth. According to the schema, the author resolver returns a User object, and the User type has a mentions resolver, so we could also fetch the tweet's author's mentions. Compare the below to our last query:

User with two levels of mentions

The mentions resolver is being used a second time at the deepest nesting level of the query response.

MongoDB

The tweet's views are stored in MongoDB, so we need to do a findOne on our Views collection, which has documents of the form:

{
    "_id" : ObjectId("5732432beca6120bbf6c0df3"),
    "tweetId" : NumberInt(1),
    "views" : NumberInt(82)
}

And we can look up the right doc by its tweedId:

Tweet: {  
  author(tweet) { ... },
  views(tweet) {
    return Views
      .findOne({ tweetId: tweet.id })
      .then(doc => doc.views);
  },
}

code

Redis

Next up we have the publicFeed, which we keep in a Redis list.

Public feed

Whenever someone tweets, we LPUSH it onto the list and LTRIM it back to three items. Then in the query resolver, we fetch the whole list with LRANGE, and since Redis just stores strings, we JSON.parse the result.

const resolvers = {  
  Query: {
    user(_, args) { ... },

    async publicFeed() {
      const feed = await redis.lrangeAsync('public_feed', 0, -1);
      return feed.map(JSON.parse);
    },
  },
  User: { ... },
  Tweet: { ... },
}

code

This works fine when you're storing a normal flat tweet with a userId. However, to improve latency and reduce load on Postgres, we store the whole user document as part of the Redis tweet:

{
  id: 1,
  text: "Est dicta ullam aliquid quod et.",
  city: "New York",
  created: 1481763442107,
  user: {
    firstName: "Pansy",
    lastName: "Herzog",
    photo: "http://placekitten.com/200/139"
  }
}

This means we need to modify the author resolver—which currently always does tweet.getAuthor())—to instead return the user subdoc if it exists:

Tweet: {  
  author(tweet) {
    if (tweet.user) {
      return tweet.user;
    }

    return tweet.getUser();
  },
}

code

REST

Lastly we have the cityFeed query—the most recent tweets in the client's city. For someone in Mountain View, this would look like:

City feed

These tweets are stored in Postgres, and we'll look them up based on the tweet.city field, but first we have to figure out the client's location. During the server setup we get the request's IP address and add it to the GraphQL context:

const graphQLServer = express();

graphQLServer.use('/graphql', bodyParser.json(), graphqlExpress((req) => {  
  const ip = req.ip;

  return {
    schema,
    resolvers,
    context: { ip },
  };
}));

code

The context is available as the final argument to all resolvers. Once we have the IP, we can make a GET request to http://ipinfo.io/${context.ip}, which returns a JSON response like:

{
  "ip": "8.8.8.8",
  "hostname": "google-public-dns-a.google.com",
  "city": "Mountain View",
  "region": "California",
  "country": "US",
  "loc": "37.3860,-122.0838",
  "org": "AS15169 Google Inc.",
  "postal": "94040"
}

which has the city attribute that we need. Putting it all together, with Sequelize's Tweet.findAll function:

import rp from 'request-promise';

async cityFeed(_, args, context) {  
  const response = await rp(`http://ipinfo.io/${context.ip}`);
  const { city } = JSON.parse(response);

  const cityTweets = await Tweet.findAll({
    where: { city },
    limit: 3,
    order: [['created', 'DESC']],
  });

  return cityTweets;
},

code

Performance

One advanced issue to note is that basic GraphQL resolver implementations like the above can result in many repeated single-record queries. You can improve latency and reduce load on the database with the DataLoader library, which batches database queries (eg for SQL batches multiple User.find({ where: { id: args.id }}) queries into a SELECT * WHERE IN query) and caches responses. That’s often sufficient, but for relational databases you can also do JOINs with Join Monster.

Done

And we're done! Here's what the final resolvers.js file looks like, and the whole repository. The only thing left out was the database/ORM setup and seeding, which you can find in connectors.js.

I hope you agree that this is a fantastic way to provide an API for your clients—one that's easy to build, painless to consume, self-documenting, and version-free. There are some aspects of a GraphQL server I didn't cover, which you can read about at graphql.org and in the Apollo server library documentation. To learn about using GraphQL on the client (React, Angular, React Native, or native mobile), check out the Apollo Client docs.

And finally, the best resource for learning GraphQL in depth will be my upcoming book, GraphQL: The New REST 👌. Sign up at graphql.guide to be notified when it's released 😄🙌.

[Loren Sands-Ramshaw](http://lorensr.me/) is an author of the upcoming book [GraphQL: The New REST](https://graphql.guide). He also freelances and helps maintain the [Meteor](https://www.meteor.com/) framework. ☄️🤓

attributionHyberbole and a half

Conquer the Data Layer

Spend your time developing apps, not managing databases.