Find or Build Rest APIs for any CMS, Framework or Service

You need to build a webpage that dynamically filters lists of items from existing data managed by a content management system. You know your way there is Javascript and AJAX. But what do you do, when there is no API?

Before you cut your losses and build one from scratch, the checklist below will help you to find out what options you have. Sorted from best to worst case: The deeper you have to investigate, the more work you will have to do yourself.

First: Make sure they really do not offer any API

Original Artwork by Commitstrip

Sometimes, the creators of the tooling you use have already thought about how front-ends consume the data. So they provide a Web API, that you can access via HTTP.

You can browse the list of common CMS and their associated API below.

CMS and Frameworks

Help improve the list

Public APIs and Services

Otherwise, if you are using a service or want to incorporate google maps or similar with your website, see this list of public APIs and how to use them.

If you could choose a CMS (which is barely the case if you came here), I would recommend a headless CMS. (Strapi is my favorite self-hosted option right now) They are also known as Content as a Service. They always expose an API, and are built for consumer-agnostic front-ends. Which means that all your UI has to speak is JSON.

Can't find any API here? Search for your CMS or Framework plus one of the keywords below:

• Rest API
• API
• AJAX
• XHR
• Fetch
• OpenAPI
• Swagger
• SDK
• Client
• GraphQL
• JSON
• XML
• SOAP (If it is ancient and runs on .NET framework)

There is also a new API type becoming more and more popular: GraphQL. GraphQL ignores the semantics of HTTP completely, but has a more efficient way of asking for data.

Second: Is there a way to expose the data via an extra route?

If there is no API, the workaround to implement it is sometimes not that far away. Sometimes all it needs, is to expose the data and encode it as JSON with the correct content-type header. Almost all CMS come with their own preferred ways of retrieving data. If all they have is templates, then you have to go deeper.

• Are there methods to retrieve the data in another format? (PHP Objects, JSON, XML)
• Can you place the data on another page than the rendered page?
• Can you add new custom routes?
• Is there some pattern like MVC in place?

Most CMS and frameworks follow a MVC pattern - Model View Controller. This means that the mechanism of retrieving the data and displaying it are somewhat uncoupled. This is your ticket to an API.

The challenge here, is to create a route and expose the data.

Here is one example for PHP and one for NodeJS:

PHP

If you can see the file endings like http://domain.com/index.php in your url, then you can simply create a file that you can ask for in your front-end. Otherwise you will have to take a look in how routing works in that CMS.

<?php
// retrieve data - check with your CMS documentation
$items = $controller->find($_GET);
header('Content-Type:application/json');
echo json_encode($items);
die;

$_GET contains all query parameters (everything after ?, like ?sort=price). You use them to filter how a request should respond. Not only for GET requests.

NodeJS

NodeJS is way less restrictive than Apache servers. In NodeJS, you do not have a static server by default.You have to create the routes on purpose. It is pretty easy to find out how to hook into existing running server.

const express = require('express')
const app = express()

app.get('/', async (req, res) => {
    let items = await Item.where(req.query).fetchAll()
    res.send(items)
}

app.listen(80)

If you have trouble with incorporating authentication or other requirements, have a look at this list of articles to build proper APIs.

Third: There is no method to retrieve the data? Build your own

We are getting deeper.

Usually this is where you want to get a backend developer on board, if you haven't already in the previous step. But as it is often the case as Web Developer, you are more than often left alone with doing everything on your own. And it doesn't stop just there with some HTML, CSS and Javascript. You don't have to do that alone.

If your software does not expose any method to retrieve the data from a file, database or somewhere else, you might as well build your own project from scratch. But here we go.

Now, you have to find out where the data comes from.

This is highly specific to both the way how data is stored and what programming language you are using to retrieve it. Usually you want to go with the database if there is already one. Be it SQL with MariaDB, Postgres or MySQL. Or for NoSQL you can keep an eye open for CouchDB or MongoDB credentials.

Most of the time you will deal with a SQL database like Wordpress comes along with.

In PHP, you can use mysqli to get the data from MySQL/MariaDB databases or for any other DB, have a look at PDO. Don't forget to use the query parameters as described above and properly escape all strings. If you do not shy away from using the terminal, you can use composer to pull in other tooling to abstract away SQL. If I'd have to recommend one, I would go with PropelORM, but there is no easy to implement option.

For NodeJS, you can easily make use of tools like bookshelf or mongoose to create models out of tables or documents. This makes for a way more convenient data access abstraction than using SQL directly.

Otherwise, if there really is no other way, see below.

Fourth: Abuse and parse the HTML

If there is no other way, you could parse the HTML.

As absurd as it sounds, I once had to do this for a professional company website. They based their content on their wiki solution (Confluence). I needed to parse the HTML contents of some pages to get data such as labels, categories and so on for their blog posts.

It might sound and feel very hacky, but guess how google delivers results and partially meaningful information with their search engine: they read the HTML.

The difficulty here is to not have the client (the browser or front-end) to parse it, but have a server relay that does the heavy lifting for you.

In the worst case, you can open another port on the same server that parses the HTML and turn it into proper JSON for you.

Cheerio is a nice jQuery-like tool to parse HTML for NodeJS server - they are quickly set up.

Conclusion

Those are some ideas to conquer the server territory and make sense of the missing pieces required to build dynamic front-ends.

Sometimes this is needed to pave the path for crafting living and breathing webpages that are not bound to page-refreshes like in the 90s. Most of the times we are bothered with software that was not built for this age of the internet, so you have to make one.

Get a sneak peak into my interactive book and find out how to make a way, no matter what backend obstacle you face.

Wire Up your Front-End