How to Schedule API Calls with Scripts and Cron

Running an API call as a scheduled task is a common need for any business. Whether you need to synchronize data between two endpoints or regularly pull information from one system to another, it makes sense to automate the process rather than remembering to do it manually every time.  As sysadmins everywhere know — you can schedule API calls with Cron.

Cron is the standard job scheduling utility for Linux and other Unix-like operating systems. Its history dates to AT&T’s Version 7 Unix from 1979. In the decades since, Cron’s flexibility, ease of use, and reliability have made it an essential tool for anyone needing to run tasks on a repeating basis.  Here, you’ll learn everything you need to know about scheduling an API call with Cron — from writing a script to creating a Cron job that runs the call as often as you want. Here’s a little about Cron, its lingo, and how it works to get you started.

Here are the key things to know about how you can schedule API calls with scripts and Cron:

• ERP systems consolidate business processes, but deployment preferences vary based on company size and cost.
• Deltek Costpoint offers comprehensive features but faces challenges like complexity and limited customizability.
• Modernizing legacy mainframes results in cost savings, enhanced performance, and better security.
• APIs facilitate seamless integration in ERPs, enhancing data sharing and automation.
• DreamFactory stands out for Costpoint integration due to its secure and flexible API-based approach.

The History of Cron

Periodically running a program or script is a common need on any operating system. The consumer-focused operating systems of today have tools built in to meet this need, like Microsoft's Windows Task Scheduler or Launch Services on macOS. But when Unix was being developed at AT&T’s Bell Labs in the 1970s, there was no precedent for this sort of thing. In 1975, Version 6 Unix became the first commercially available Unix OS, but it didn’t offer users a way to configure jobs to run on an automated schedule.

Four years later, Version 7 Unix shipped with the first public release of Cron. While the version of Cron that ships today with Linux distributions and macOS is not complicated, Cron in V7 Unix was extremely basic. It ran once a minute, read from a table of jobs located at /etc/lib/crontab, and determined which tasks needed to be run at that time. Subsequent versions have added several features. Perhaps the most important was a move away from a central list of jobs to be run, enabling each user on a system to have their own job files (Cron tables, or crontabs for short).

It's generally accepted that the name "Cron" refers to the Greek word Chronos, which translates to “time.” Clocks and other time-keeping systems often use the “chrono-“ prefix. Since the original help for Cron refers to the app as “cron — clock daemon,” this theory seems to fit.

How to Use Cron

Cron is a package that consists of two parts: the cron daemon (a process that runs in the background) and the crontab utility. The latter is how a user interacts with the file that contains the list of tasks to be run along with their scheduled times.

One big difference between the original Cron and today's implementations is that all tasks run with the permissions of the user that scheduled them. In V7 Unix Cron, all scheduled jobs ran as root — the superuser that can perform any task on a Unix system. The potential for security risks and damage to another user's files makes running Cron jobs as root a rarity today.

When you need to schedule API calls with Cron, you will run the crontab utility, and you need to open a terminal application to access the command line. In most cases, a user will invoke crontab with the "-e" argument like this:

crontab -e

This allows the user to edit their crontab. Some other arguments that can be used are "-l", which lists the contents of the user's crontab, or "-r", which deletes the user's crontab.

When you run crontab -e your default terminal text editor loads your crontab file. Your default editor is defined with the $EDITOR environment parameter.

Once your crontab is loaded, Cron jobs are created on a single line with six fields. The first five fields define the times at which the job should run:

1. Minutes past the hour, with values from 0 to 59.
2. Hours, from 0 to 23.
3. Days of the month, from 1 to 31.
4. Months, from 1 to 12.
5. Days of the week, from 0 to 6. 0 is Sunday, and 6 is Saturday.

An asterisk can be used as a wild card in any of these first five fields. For example, an asterisk in the Days of the Month field will run daily.

The sixth field contains the path to a script or a program to be run.

Here are a few examples, with the syntax explained:

15 * * * * /path/to/script

This crontab entry would run the script at 15 minutes past every hour, every day of the month, and every day of the week.

The first five fields allow the use of hyphens or commas to denote multiple values in the field. To run the script at each of the first ten minutes of every hour.

0-9 * * * * /path/to/script

And this runs the script at 15 minutes and 30 minutes past each hour.

15,30 * * * * /path/to/script

Once you have added your entries for each schedule in your crontab, save and close the file. The Cron daemon now runs your jobs at the schedules you've set.

Next, you need a script with an API call that you can schedule API calls with Cron.

Scripting an API call

This tutorial uses a PHP script that synchronizes records between two databases via API calls. This is taken from the DreamFactory docs, but you are not limited to using PHP. A wide variety of other scripting languages, like Python or JavaScript, could be used. This script was chosen for its simplicity, and the comments will help you understand how API calls can be integrated into scripts. More complicated operations, like authentication and notifications, can be scripted if you need to.

This example script takes records from a table named employees in a MySQL database via API and allows you to send the data to another database API. The receiving end could be MySQL, SQL Server, or any other database server with REST API access. It can keep data synced in real-time depending on how often you run the script.

Remember to include your shebang at the top of any script you write. This tells your system what language you are using and ensures the script will be interpreted properly. As we are using PHP, our shebang will point to the PHP interpreter:

#!/usr/bin/php

This is a common location where PHP is found, but to be sure where it's located on your system, you can run this command in a terminal:

which php

Now that you're ready, here's the example script:

#!/usr/bin/php

// Assign the $platform['api'] array value to a convenient variable

$api = $platform['api'];

// Declare a few arrays for later use

$options = [];

$record = [];

// Retrieve the response body. This contains the returned records.

$responseBody = $event['response']['content'];

// Peel off just the first (and possibly only) record

$employee = $responseBody["resource"][0];

// Peel the employee record's first_name and last_name values,

// and assign them to two array keys named first and last, respectively.

$record["resource"] = [

    [

        'first' => $employee["first_name"],

        'last' => $employee["last_name"],

    ]

];

// Identify the location to which $record will be POSTed

// and execute an API POST call.

$url = "contacts/_table/names";

$post = $api->post;

$result = $post($url, $record, $options);

exit

Now, save the script as database_sync.php. Let's assume your username is "me," and you are saving the file in your home folder, making its path /home/me/database_sync.php

This example path will be used throughout the rest of this article.

Before you can schedule your API call to run with Cron, you need to make the script executable. This tells your system that running the code in your script is OK. In a terminal window, you can use the chmod command to add the executable bit for your user:

chmod u+x /home/me/database_sync.php

After that, ensure the script runs as expected and if not, do some basic troubleshooting. You are now ready to schedule your API call.

Schedule API Calls with Cron

All that's left to do is to edit your crontab and add a schedule for your API call script. In your terminal window, open up your crontab for editing:

crontab -e

Then add a line like this:

0,15,30,45 * * * * /home/me/database_sync.php

This will run your API call script at the top of every hour and then again every 15 minutes.

This example will walk you through creating a CRON job to synchronize your data between two databases every minute. This of course can be modified to run on different intervals and used for other tasks. 

And that's it! You have now successfully scheduled an API call. Although this script was just an example, its query parameters and request body can be used as a template.

API Management the Right Way with DreamFactory

Exchanging data between systems via APIs is a critical component of the modern digital enterprise. Whether you need to schedule API requests with the help of Cron or you're making a one-time call, you need a first-class API management platform. DreamFactory and our iPaaS (internet Platform as a Service) are here to help you manage your API workflow needs. Start your free 14-day trial.

Check out "Scheduled Tasks with DreamFactory".

Frequently Asked Questions - Using Cron for Task Scheduling and API Calls

What is Cron, and why is it important for task scheduling on Unix-like systems?

Cron is a system that enables automated execution of tasks or scripts on Unix-like systems at specified intervals, providing a solution for scheduled task management.

How does Cron differ from modern task scheduling tools on other operating systems?

Cron emerged as a solution for task scheduling on Unix systems before modern tools existed, allowing users to automate tasks at specific times or intervals.

How do I define the schedule for a Cron job?

A Cron job schedule consists of six fields, defining minutes, hours, days of the month, months, days of the week, and the path to the script or program to be run.

Can I schedule API calls using Cron?

Yes, Cron can be used to schedule API calls for tasks like data synchronization between systems. An example using a PHP script is provided in the article.

How can I edit my crontab to add or modify scheduled tasks?

Use the crontab -e command to open and edit your crontab in the default text editor, allowing you to add, modify, or remove scheduled tasks.

Can I run tasks with different user permissions using Cron?

Yes, modern implementations of Cron allow tasks to run with the permissions of the user who scheduled them, improving security compared to earlier versions.

What should I consider when troubleshooting issues with Cron jobs?

If your Cron job isn't working as expected, ensure your script is executable, review the syntax of the Cron job entry, and check system logs for error messages related to the Cron job.