Theophilus Ngaribvume

This is where i share my ideas, and thoughts.


GatsbyJS Create your own Source API Plugin

08 June, 2021 - 4 min read

GatsbyJS Create your own Source API Plugin

Learn how you can create your own gatsby source api plugin from scratch to meet your API demands. We'll build a real API you can use on your production server.

A bit history of my motivation creating a source plugin, i have a remote jobs aggregator Workly, This project fetches jobs from stackoverflow, remoteok, remotive, and adzuna jobs. All these sources have a different way of returning the jobs:

  • Remoteok returns the data on data node
  • Remotive returns the data on data.jobs

Working with Gatsby GraphQL API, i wanted to have the ability to filter jobs depending on the categories, and tags. For remoteok it was simple, but for remotive was a bit difficulty. So i decided to create a source plugin for remotive jobs API.

Let's start

A gastby plugin is created as an npm package or local plugin. The plugins is required to define it's own, node, server-side rendering and the browser Gatsby APIs.

The plugin will be configure in the site's gatsby-config.js file.

I'm going to assume you have already setup a gatsby website and ready to build a source plugin for your site. In the following example, we're going to create a remotive jobs api source plugin.

Set up the source plugin file stucture.

You'll create the plugin with a gatsby new command, And we're going to use the plugin starter. Navigate to your root project folder and run the following command.

gatsby new remotive https://github.com/gatsbyjs/gatsby-starter-plugin

The above command is going to create a remotive folder inside your main project.

gatsby-node.js

gatsby-node.js is where all the Gatsby node APIs are used for customisation. All the logic of the source plugin are found in this file.

Add the following code to the remotive/gatsby-node.js.

// constants for your GraphQL Post and Author types const fetch = require("node-fetch") const POST_NODE_TYPE = `remotive` exports.sourceNodes = async ({ actions, createContentDigest, createNodeId, getNodesByType, }) => { const { createNode } = actions const data = await fetch("https://remotive.io/api/remote-jobs").then((res) => res.json() ) // loop through data and create Gatsby nodes data.jobs.forEach((job) => createNode({ ...job, id: `${job.id}`, parent: null, children: [], internal: { type: POST_NODE_TYPE, content: JSON.stringify(job), contentDigest: createContentDigest(job), }, }) ) return }

Using the plugin.

After defining the plugin gatsby-node.js, now is the time to use it inside your site.

Open your main site gatsby-config.js, where you install other plugins and add.

{ resolve: require.resolve(`../remotive`), },

Run your gatsby project, and you should be able to see the remotive nodes added. Now you can use your remotive jobs data as you do any other data in gatsby.

Customise plugin usage

As you have seen, the plugin usage is rigid, imagine having multiple data sources that you want to impliment the same way we did the remotive plugin. So to customise the plugin we're going to seperate two things from implimentation and define them inside the website main gatsby-node.js file. .

  • POST NODE TYPE
  • Source URL

To do that, we need to define the option inside our plugin configuration.

{ resolve: require.resolve(`../remotive`), options: { apiURL: "https://remotive.io/api/remote-jobs", name: "remotive" }, },

Now, we can utilise, apiURL and name inside our plugin gatsby-node.js configuration file.

Open remotive/gatsby-node.js and edit as follows.

// constants for your GraphQL Post and Author types const fetch = require("node-fetch") exports.sourceNodes = async ( { actions, createContentDigest, createNodeId, getNodesByType }, pluginOptions ) => { const { createNode } = actions const data = await fetch(pluginOptions.apiURL).then((res) => res.json()) // loop through data and create Gatsby nodes data.jobs.forEach((job) => createNode({ ...job, id: `${job.id}`, parent: null, children: [], internal: { type: pluginOptions.name, content: JSON.stringify(job), contentDigest: createContentDigest(job), }, }) ) return }

Now you can install the plugin to fetch as many endpoints as you want. Let's rename our fetch plugin into a more generic name since we're now fetching from multiple endpoints. I suggest renaming to api-sources, so rename your remotive folder to api-sources.

{ resolve: require.resolve(`../api-sources`), options: { apiURL: "https://remotive.io/api/remote-jobs", name: "remotive" }, }, { resolve: require.resolve(`../api-sources`), options: { apiURL: "https://example.com/api", name: "examplejobs" }, },

Happy Coding !

Creator of Workly and Nerdlify, Currently working at Pindula.

© 2021, Theophilus Ngaribvume. All Rights Reserved.