Now that we have explored monitoring apis with Prometheus, lets take a look at monitoring our APIs with Grafana. You may have noticed from the previous blog that Prometheus is awesome, but takes some time to fully flesh out and their dashboards aren’t the best. Good thing Grafana specializes in data visualization. We will not even have to configure all our Prometheus queries, so let’s get started.
We first need to confirm everything is running, if so your output should look similar to the above image. If you are not familiar with getting everything up and running please reference our previous blog. By navigating to 127.0.0.1:3000 you should see the Grafana home page. If you are not familiar with Grafana you may be wondering where the dashboards are. They are all located in the top left dropdown where we will see 4 pre-configured dashboards. Let’s review each of them to understand what each of them does.
Docker Host Dashboard
The Docker Host Dashboard shows metrics for monitoring the usage of the server. Thanks to dockerprom(link github repo) for pre-configuring all this we can now see:
Server uptime, CPU idle percent, number of CPU cores, available memory, swap and storage.
System load average graph, running and blocked by IO processes graph, interrupts graph.
CPU usage graph by mode (guest, idle, iowait, irq, nice, softirq, steal, system, user).
Memory usage graph by distribution (used, free, buffers, cached).
IO usage graph (read Bps, read Bps and IO time).
Network usage graph by device (inbound Bps, Outbound Bps).
Swap usage and activity graphs.
Docker Containers Dashboard
The Docker Containers Dashboard shows key metrics for monitoring running containers:
Total containers CPU load, memory and storage usage.
Running containers graph, system load graph, IO usage graph.
Container CPU usage graph.
Memory usage graph.
Cached memory usage graph.
Network inbound usage graph.
Container network outbound usage graph.
Monitor Services Dashboard
The Monitor Services Dashboard shows key metrics for monitoring the containers that make up the monitoring stack:
Prometheus container uptime, monitoring stack total memory usage, Prometheus local storage memory chunks and series.
Container CPU usage graph.
Memory usage graph.
Prometheus chunks to persist and persistence urgency graphs.
Chunks ops and checkpoint duration graphs.
Samples ingested rate, target scrapes and scrape duration graphs.
We now have a fully functioning monitoring tool with no code. Isn’t that awesome? A no code API tool and a no code data visualization tool, it can’t get much better than this. This post has only scratched the surface with regards to what you can accomplish with these tools. If you are interested in this for a Ruby application, check out our friends over at Scout who wrote a blog about doing exactly that. If you like what you see, sign up for a free trial today or contact us for more information!
There is no question DreamFactory’s native connectors have saved IT teams countless hours of development time. Yet these are almost incidental when one takes into consideration the platform’s ability to integrate with thousands of third-party REST and SOAP services, not to mention create entirely new APIs through the scripted service interface. Further, thanks to DreamFactory’s ability to leverage third-party libraries, new APIs can often be created in just a few dozen lines of code.
Continue reading “Creating a Geocoder Service Using DreamFactory and the Google Maps Geocoding API”
Integrating Third-Party Libraries With DreamFactory Scripting
Being able to script business logic into your endpoints to further extend and customize the data response(s) and request(s) is one of the best features inside of DreamFactory. As a form of ETL data integration, DreamFactory can help you create a data lake or data warehouse, especially when combined with a service such as Xplenty. Sometimes cases arise where you need to take advantage of the robust third-party library community to help extend your API. In the example below, we are going to use a third-party library to help us connect our DreamFactory instance to a Twitter Account to post updates. We also bring in the conecpt of connecting the Twitter service to our Rapsberry Pi IoT device to post temperature information to show you how can connect multiple services through DreamFactory to manage your data needs in a simple, robust method that makes API mangement a snap.
DreamFactory has had support for Logstash since version 2.3 for our Gold Tier version. Elastic makes some great tools to support very robust logging. Incorporating Elasticsearch, Logstash and Kibana into your powerful, scalable DreamFactory instance is a no brainer, especially for users who have a lot of data being pushed and pulled through various endpoints. This will make the lives of your admins so much easier with the amount of detail they can grab to troubleshoot issues.
Continue reading “How To Configure An ELK Stack With DreamFactory”
DreamFactory 2.7 introduces a new system endpoint – api/v2/system/import that allows you to import data files using a database service of your choice. Currently the feature supports CSV files. Support for XML and JSON is on our roadmap for future releases. This new endpoint is a DreamFactory native endpoint and it is a part of the “system” service. This endpoint is available to use right out of the box without the need for installing additional driver/extensions.
DreamFactory 2.7 has shipped! This release includes support for CSV file import, OpenID Connect, a new installer program to customize your DreamFactory installation, and many more bug fixes and enhancements. Get the 2.7 release from Bitnami or GitHub now. Here’s what’s new:
Twilio has a superb API for integrating SMS messages into your applications. It’s easy to add Twilio as a remote HTTP service to any application you’re building with DreamFactory. DreamFactory lets you securely store your Twilio authentication credentials, call the Twilio API directly from a DreamFactory session, and easily add role-based access control to any Twilio API endpoint. This brief tutorial shows you how to add Twilio to DreamFactory in five minutes.
Building and deploying data-driven applications, both web and mobile, typically requires a handful of development resources. Consider an enterprise application team consisting of a DBA, server-side team, client-side team, mobile team, and IT resources. Coordinating the work among team members and piecing together the front-end and back-end components of even a basic data-driven application is hard. But it shouldn’t be.
The good news is that there are many client technologies today that make front-end mobile application development a lot easier, particularly if you’re transitioning from web application development to “responsive” mobile app development. These front-end technologies include HTML5 for any device, great new client frameworks like Twitter Bootstrap and AngularJS , “hybrid” app technologies like Adobe PhoneGap and Sencha, and native development SDKs for iOS, Android, and Windows.
What’s missing is an easy way to create the back-end of your mobile application. Today the heavy lifting is done by server-side engineers. That means configuring servers and back-end software, building back-end service interfaces, and testing front-end and back-end integration. These are all time consuming and complex tasks, even for a simple mobile application.
We believe developing a mobile application from scratch should be dramatically easier for front-end developers. You shouldn’t have to become a “full stack” engineer to code and deploy a world-class mobile application. Taking the back-end roles out of the equation and enabling a front-end developer to build and deploy their work makes life alot easier. It’s not overwhelming.
We built the DreamFactory Services Platform (DSP) to help address this challenge. More on that in our next blog post.