How I was port-out scammed in 3 hours

Tokyo, midday on a Friday. My phone kept buzzing during my customer meeting. The day was a workday, a day of travel and the day a personal property purchase was due to settle. Little did I know it was also the day I was the target of a “port-out” scam.

In the DreamFactory Tokyo office in Azabu Juban, Minato, in the few minutes I had between one meeting and the next, I glanced at my phone trying to keep pace with solicitor updates regarding my property transaction finalizing. Amidst the deluge of notifications my phone was providing, one thing I quickly dismissed were two verification SMS codes sent by my telco provider that I hadn’t asked for and didn’t give a second thought to:

Continue reading “How I was port-out scammed in 3 hours”

DreamFactory News – Training, a New Guide, SaaS, and More

DreamFactory Jeanie
Our New Mascot. Say Hello to Jeanie. Stay Tuned For More

DreamFactory Happenings

It’s hard to believe the year’s end is almost upon us! This has been a pretty transformative year for the company. We’ve seen record demand for the DreamFactory Platform, and have additionally been working around the clock on a number of new initiatives: Continue reading “DreamFactory News – Training, a New Guide, SaaS, and More”

Improved Data Security with MySQL Privileges and DreamFactory

DreamFactory and MySQL

All MySQL installations naturally include a root account and offer the ability to create restricted user accounts. However, otherwise sane developers will often use these root accounts for application-level communication, dramatically raising the likelihood of data theft, data exfiltration, and other security issues. For that reason the DreamFactory team always recommends users take care to create restricted MySQL users before using the platform to generate APIs.

In this tutorial, you’ll learn how to create a non-root MySQL user and then further restrict this user’s privileges to a specific database and even table subset. You’ll also learn how to subsequently revoke a user’s privileges to reflect changing requirements.

Continue reading “Improved Data Security with MySQL Privileges and DreamFactory”

Learning About The Bitnami System Database

Database Code Lines

The Elusive Bitnami System Database

If you want to spin up a fast API solution, DreamFactory is a great way to do that with a Bitnami install. Within minutes you can have a fully documented and secure REST API to utilize. Just like any program bundle, there are lots of features to learn and interact with.  Outside of a Docker Swarm or AWS ELB setup, it is pretty hard to find a way to spin up a DreamFactory instance faster. We are going to dive in a bit further to find out how to interact with the system database. Continue reading “Learning About The Bitnami System Database”

SOAP vs. REST APIs – Which Reigns Supreme?

SOAP vs REST
SOAP vs REST
These days it’s more true than ever that “no company is an island”. Many businesses rely on each other by exchanging information and much of that exchange is done via an API. When diving deeper into the question of developing APIs, you’ll undoubtedly encounter the question: SOAP or REST? Although REST APIs have become the most popular choice for today’s businesses, the decision isn’t always an easy one. In this article, we’ll go over everything you need to know about SOAP and REST APIs. That way you can come to the conclusion that’s ultimately right for your situation.

What is an API?

There are a lot of definitions of the term “API” out there, and many can be confusing. An API (Application Program Interface) is a way for you to get information needed in a consistent format. You can think of an API as like an interaction between a business and a customer, such as placing an order at a restaurant.
  • Customers first read the menu, then they decide what food they would like to order.
  • The waiter serves as the “middleman” between the customer and the business. They take the request and present it to the business in a way that’s most comprehensible and efficient.
  • The business reviews the request and sends back a response to the customer, such as a plate of food.
It’s important to note that in this example, the interaction is entirely predictable. When customers go to a restaurant, they can assume that they’ll be handed a menu. Then use that menu to place an order, and receive the food they ordered. In the same way, APIs offer consistency to users who want to query a website for its data. By establishing a common set of rules for exchanging information, APIs make it easier for two parties to communicate.

What is a SOAP API?

SOAP (Simple Object Access Protocol) is an API protocol that uses the XML Information Set specification in order to exchange information.

SOAP vs REST

A standard SOAP message consists of the following XML elements:
  • An Envelope element that identifies the document as a valid SOAP message
  • An optional Header element that specifies additional requirements for the message, such as authentication.
  • A Body element that contains the details of the request or response.
  • An optional Fault element that contains information about any errors encountered during the API request and response.

What is a REST API?

REST (Representational State Transfer) is an architectural style for APIs that relies on the HTTP protocol and JSON data format to send and receive messages. REST utilizes CRUD (Create, Retrieve, Update, and Delete) to keep API calls as simple as possible to understand. With its simplicity, developers can understand it whether they are the lead architect or a junior developer just starting.

SOAP vs REST

As highlighted by BBVA some major advantages to REST Visibility, reliability and scalability. REST is very simple to scale and can be reused across several projects. This provides you with the advantage of faster development cycles with far less headaches. It also is independent from being bound to a single language, meaning you can use it with any language such as PHP, JavaScript, or Python.

The Pros and Cons of SOAP and REST

When comparing REST and SOAP, people often use the analogy of a postcard and an envelope. REST is like a postcard in that it’s lightweight and consumes less bandwidth (paper). Meanwhile, SOAP is like an envelope: there’s an extra overhead required on both ends to package and un-package it. Note that the analogy isn’t perfect: unlike a postcard, the content of REST requests and responses isn’t (necessarily) insecure. Instead, REST uses the security of the underlying transport mechanism, which is usually HTTPS. On the other hand, SOAP implements its own security measure, which is known as WS-Security. Some people believe that REST is largely a “replacement” for SOAP, due to its lower overhead and improved ease of use. According to Cloud Elements’ 2017 State of API Integration report, 83 percent of APIs now use REST, while only 15 percent continue to use SOAP. Some of these businesses primarily use REST, but continue to integrate SOAP APIs into their projects using tools such as DreamFactory’s SOAP connector. However, the thought of SOAP being outdated isn’t quite accurate. Even as REST becomes the API style of choice for most businesses, SOAP remains a tool that is better-suited for certain use cases, mainly in large enterprises who need additional extensibility and logic features native to the protocol.

The advantages of REST include:

  • Flexibility: Although REST is most commonly implemented with HTTP and JSON, developers are by no means obligated to use them. Websites can send back responses using data formats including JSON, XML, HTML, or even plaintext-whatever best suits their needs.
  • Speed: Because it tends to use much less overhead, REST APIs are typically significantly faster than SOAP. While the differences might be imperceptible for a single request, the disparity grows larger and larger as you place more and more requests.
  • Popularity: REST has reached critical mass on the Internet. Major websites such as Google, Twitter, and YouTube all use REST APIs for users to send and receive messages. Due to this familiarity, it’s typically easier for developers to get up and running with REST.
  • Scalability: Thanks to their speed and simplicity, REST APIs usually perform very well at scale.
Despite the major benefits of using REST, SOAP remains the preferred protocol in certain use cases. Some organizations find that SOAP offers the transactional reliability that they’re looking for, while others simply continue to use SOAP because they need legacy system support.

The advantages of SOAP include:

  • Formality: SOAP can use WSDL (Web Services Description Language) to enforce the use of formal contracts between the user and the website. SOAP is also inherently compliant with ACID database standards, which ensures that the transactions it performs will be valid even in the event of errors or hardware issues.
  • Logic: If a REST API request is unsuccessful, it can only be addressed by retrying until the request successfully goes through. On the other hand, SOAP includes built-in successful/retry logic so that the requesting system knows how to behave.
  • Security: SOAP comes with its own security mechanism, WS-Security, built into the protocol. If you want to ensure that your messages are secure, rather than relying on the underlying transport mechanism as does REST, then SOAP may be the right choice.
  • Extensibility: In addition to WS-Security, SOAP includes support for other protocols such as WS-Addressing and WS-ReliableMessaging that can define other standards of communication and information exchange.

Converting SOAP to REST


SOAP to REST

DreamFactory can convert your SOAP service to REST in minutes. If you have your SOAP service RESTified you can take advantage of the features DreamFactory provides. Like turning security up 10 notches by securing the API with API Keys and Roles to only allow access to people that have the right to it. Even add custom events and business logic by way of our powerful Scripting engine.

Conclusion

For most cases, REST should be considered the “default” option as adoption continues to grow across the web. Most public-facing APIs now use REST, because it consumes less bandwidth and its compatibility with HTTP makes it easier for web browsers to use. However, you may find that the additional features and security offered by SOAP are enough to sway your decision. In the end, the “right” choice between SOAP and REST will be highly dependent on your own situation. Even better, the choice of SOAP and REST doesn’t have to be between one and the other. If you want to communicate with REST but still need access to legacy SOAP services, DreamFactory offers the ability to add a REST API onto any database or SOAP API. See how quick and easy it is!


If you have your SOAP service RESTified we can take advantage of the features DreamFactory provides. Like turning security up 10 notches by securing the API with API Keys and Roles to only allow access to people that have the right to it. Even add custom events and business logic by way of our powerful Scripting engine. If you know REST APIs are the best for you, don’t waste any more time building them. Try it yourself with your own instance of DreamFactory or connect with an API specialist today!

AWS Redshift – SQL Functionality on Planet-Scale Hardware

The Problem

Your manager’s peers have been bragging a lot lately about their data warehouses, analytics, and charts, and now a steady stream of data-related questions are being sent your way.  Your department maintains several databases, and the data they contain has the potential to answer everything management is asking for. But the databases are needed for day-to-day operations, and can’t scale to answer these often highly specific questions such as, “How many asparaguses were consumed by men named Fonzie in Cleveland on Tuesdays in 2013?”. How to unlock the potential of this data?

You’ve probably heard of data warehouses, which are tailor-made for this sort of witchcraft. They make it possible to unlock every bit of value from data, and find answers wickedly fast. In the past, creating and maintaining data warehouses meant large, ongoing investments in hardware, software, and people to run them. This would be a hard sell – isn’t the company already spending enough?! Good news, however! In this day of cloud computing, it’s incredibly simple to create, load, and query data warehouses. They typically charge on a usage basis, meaning you don’t need the initial upfront capital investment to get off the ground. And they are super fast – far more powerful than anything you could run in-house.

This post will focus on Amazon Web Services Redshift (Amazon Web Services = AWS). And as a bonus, I’ll demonstrate the incredible Dreamfactory, which automatically builds a slick REST API interface over the top. From there, you’re a GUI away from giving management everything they could ask for, and wowing them with extras they hadn’t even thought of. They can now stand tall amongst their fellow executives, knowing you have their back.

AWS Redshift

AWS Redshift is built upon PostgreSQL, but has been dramatically enhanced to run at “cloud scale” within AWS. There are a few ingredients to this secret sauce:

Column-oriented storage

While you don’t need a deep understanding of what’s happening under the hood to use it, Redshift employs a fascinating approach to achieve it’s mind-boggling performance. Let’s say you have data that looks like the following:
ID NAME CREATED DESCRIPTION AMOUNT

1 Harold 2018/01/01 Membership 10.00

2 Susan 2017/11/15 Penalty 5.00

3 Thomas 2016/10/01 Membership 8.00
Most SQL databases you’ve probably used in the past are row-based, which means they store their data something like this:
1:Harold,2018/01/01,Membership,10.00;

2:Susan,2017/11/15,Penalty,5.00;

3:Thomas,2016/10/01,Membership,8.00;
This is the efficient way to maximize storage, and works well for retrieving data in the “traditional fashion” (rows at a time). But when you want to slice and dice this data, it doesn’t scale very well. If you’ve got large (business-scale) volumes of data, and a variety of ways you want to query it, you can really start to strain your database. Column-based databases, on the other hand, flip this idea on its head, and store the information in a column-based format, with the *data* serving as the *key*. So the above might look something like this:
Harold:1;Susan:2;Thomas:3;

2018/01/01:1;2017/11/15:2;2016/10/01:3;

Membership:1,3;Penalty:2;

10.00:1;5.00:2;8.00:3;
This drastically improves query performance. For example, when searching for “DESCRIPTION == ‘Membership'”, the query only needs to make one database call (“give me the items with a ‘DESCRIPTION’ of ‘Membership'”), instead of inspecting each row individually (as it would have to do in a traditional, row-based database). Very cool, very fast!

Massive Parallelization

When I picture what the AWS cloud must look like, I usually conjure something up from the Matrix (except it’s full of regular computers, rather than, well, humans). Or maybe Star Trek’s “Borg”, a ridiculous planet-cube flying through space, sucking up other civilizations. I guess both of those images are a little disturbing. A safer mental image is this – data centers spanning the globe, loaded with racks and racks of computers, all connected and working together. For most computing tasks, throwing more hardware at the problem doesn’t automatically increase performance. There are bottlenecks that remain in place no matter how many processors are churning away. In our “traditional database” example, this bottleneck is typically disk I/O – the processors are all trying to grab data from the same place. To overcome this, the architecture and storage have to be arranged in a way that can benefit from parallelization. Which is exactly the case with AWS Redshift. Due to the column-based design described above, Redshift is able to take full advantage of adding processors, and it’s almost linearly scalable. This means if you double the number of computers (“nodes”, in Redshift-speak), the performance doubles. And so on. Combine this scalability with the ridiculous number of computers AWS has at it’s disposal (specifically, several Borgs-worth), and it’s like staring out at a starry night. It goes on forever in all directions.

How this works for you

If you’re sold on the power of AWS Redshift, then you’ll be pleased to learn that setup is incredibly simple. AWS documentation is top notch, a crucial thing in this brave new world. When writing this post, I followed their tutorial, and it all went smoothly. Probably took me 15 minutes, and I had the example up and running. If you already have SQL expertise, you won’t have any problem picking up Redshift syntax. There are some differences and nuances, but the standard “things” (joins, where clauses, etc) all work as expected. I typically use Microsoft’s SQL Server Management Studio (SSMS), and was able to connect to Redshift with no problem (after setting it up as a linked server). Your favorite SQL client will presumably work here as well (anything that supports JDBC or ODBC drivers). Once you get your feet wet, there are myriad tools that will load your business data into Redshift. If you’ve got SQL chops in house, I’d start with the AWS documentation, and go from there. If you need a little (or a lot) of help, a whole ecosystem of companies and tools have sprung up around Redshift. A quick Google search will introduce you to them. When you’re up and running, and growing more comfortable demanding more from the system, AWS makes it incredibly simple to add capacity. Thanks to the brilliant Redshift architecture, you just add nodes, and AWS takes care of the rest. Their billing dashboard will show you what it’s costing in real time, with no hidden or creeping costs of data centers, hardware upgrades, things going bump in the night, etc. So much magic happening under the covers, and you get the credit. The joys of cloud computing!

My Humble Example

When writing this, I used the example AWS provides (it consists of a few tables containing some fake Sales data). With everything in place, I can query from SSMS (with a little bit of “linked server” glue syntax):
exec ('-- Find total sales on a given calendar date.

SELECT sum(qtysold)

FROM sales, date

WHERE sales.dateid = date.dateid

AND caldate = ''2008-01-05'';') at redshift

sum

--------------------

210

(1 row affected)
I get a thrill when a chain of systems, architectures, and networks all flow together nicely. Somewhere in a behemoth of a data center, a processor heard my cry, and spun out this result in response. Amazing.

DreamFactory

Now that the company has access to the data, and can gleefully ask any question, they are going to want the dashboards and pretty graphs. Typically you’d use a REST API to feed the data to some sort of UI, but how to do this with Redshift? While management is tickled with their new toy, they will cloud over with suspicion if you now propose a months-long project to make it shinier. In keeping with the theme of “easy, automatic, and powerful”, I’d propose using DreamFactory. In a matter of minutes (literally), it will connect to a data store (both SQL or NoSQL), intelligently parse all the schema, and spin up a REST API layer for doing all the things (complete with attractive documentation). What used to take a team of developers months can now happen in an afternoon! Here are some screenshots of my REST API, completely auto generated from the Redshift example above. It took me about 15 minutes (12 of those spent poking around the documentation) to get this done. For my simple example, I followed their Docker instructions, and in no time was playing with the REST API depicted below:
let’s get our rest on!
 
what pretty documentation you have!
  Powerful stuff!

To Infinity and Beyond!

Now that you’ve witnessed how easily you can warehouse all your data, and bootstrap it into a REST API, it’s time to bring this to your organization. Play with it a little, get comfortable with the tools, then turn up the dials.   Want to learn more about how DreamFactory and Redshift can work together (or how to put a REST API on any database)?  Schedule a demo with us. The next time management comes calling for data, you can give it to them with a fire hose!

Microsoft Server 2012 R2, SQL Server 2016 and DreamFactory – A Match Made in Heaven

Part 1: Running Microsoft Server 2012 and SQL Server on AWS, on my MacBook Pro

How do we get from here, hosting an AWS Microsoft Server instance on my MacBook Pro?
AWS Microsoft Server 2012 R2 Desktop
AWS Microsoft Server 2012 R2 Desktop
To here using Microsoft Server, SQL Server, and Dreamfactory, still on my MacBook Pro.
SQL Server Get Schema
SQL Server Get Schema

Some Background:

Let’s get to the nuts and bolts of this.  In the past, it was very difficult to cross over platforms and create Microsoft based solutions or Linux based solutions on the other’s platform.  With the advent of cloud computing, this has become increasingly easier to do. When you have a robust piece of middleware software, such as DreamFactory which is for most intents and purposes language and platform agnostic, you really do have your choice of platforms to install it on.  Each has its advantages and disadvantages, which I am not going to go into detail in this article, but suffice it to say, there are a lot of enterprises that choose the Microsoft platform(s), and some of those advantages became apparent as I worked on this post. First things first, make sure to grab all of the pre-requisites you need to make the install easy:

Required Software and Extensions

At a minimum, you will need the following software and extensions installed and enabled on your system in order to successfully clone and install DreamFactory 2.12.0+.
  • PHP 7+ – check and install the requirements below for your particular environment.
    • PHP required extensions: Curl, MBString, MongoDB, SQLite, and Zip. You may need to install other extensions depending upon DreamFactory usage requirements. If you don’t plan on using MongoDB, please remove the df-mongodb requirement from,composer.json or include the --ignore-platform-req option when running composer install.
  • Git
  •  Windows Git Client – Git Bash lets you run “Linux style” commands
  • A web server such as NGINX, Apache, or IIS. You may use PHP’s built-in server for development purposes.
  • One of four databases for storing configuration data: MS SQL Server, MySQL (MariaDB or Percona are also supported), PostgreSQL, or SQLite.
  • Composer – may require cURL to be installed from particular environment below.
Microsoft Server can be spun up almost anywhere now, as is evidenced by the photos above, and since DreamFactory is platform agnostic, we can install it on the Microsoft Server 2012 R2 instance with just a few bits of software installed to get up and running. There are multiple ways to grab and install PHP on a Microsoft platform, but an easy way is to utilize the Web Platform Installer (version 5.0 as of this post).

The Install:

You can download the Web Platform Installer for IIS here. Select a PHP version (7.0.x is required to run the current 2.13.0 version of DreamFactory), and different pieces of IIS, should you decide to utilize that as your production web server.  This post will not dive into the nitty-gritty of IIS, but you can see our documentation here.  We will be using PHP’s built-in development web server to just illustrate the connections.
Web Platform Installer 5.0
Web Platform Installer 5.0 Showing PHP installed
Once you have installed PHP and double checked your pre-requisites are installed, you can begin the install:
  • Perform a Git clone into this directory for Dreamfactory:
git clone https://github.com/dreamfactorysoftware/dreamfactory
Git Clone DeramFactory
Clone down the latest version
This will pull down the master branch of Dreamfactory into a directory called ./dreamfactory.
  • Navigate to the dreamfactory directory and install dependencies using composer. For production environment, use --no-dev, otherwise discard that option for a development environment. If you are not running or plan to run MongoDB, add —ignore-platform-reqs:
composer update --ignore-platform-reqs --no-dev
composer update --ignore-platform-reqs --no-dev
composer update –ignore-platform-reqs –no-dev
Otherwise, run the following command to install the dependencies:
composer install --no-dev
  • Run DreamFactory setup command-line wizard. This will set up your configuration and prompt you for things like database settings, first admin user account, etc. It will also allow you to change environment settings midway and then run it again to complete the setup.
php artisan df:setup
DF:Setup
php artisan df:setup
Follow the on-screen prompts to complete the setup.
Prompts
Follow the prompts
You can then run php artisan serve and migrate over to the address and port you have set up. In this example, we are running off of http://127.0.0.1:8000
php artisan serve
php artisan serve

Part 2:  The SQL Server Reckoning

With our instance running now, we can finally delve into the “fun” part of this install.  The ease with which you can add a SQL Server instance is awesome.  It is the fastest install I have ever done from the driver install to DreamFactory connection, it was less than 5 minutes¹. Using our trusty Web Platform Installer friend, you can download a SQL Server driver package that is compatible with your PHP version and your O/S version.
SQL Server Driver Package, version 5.2
SQL Server Driver Package, version 5.2
Now you can head back over to your instance and create a SQL Server service.  Just select the service type, add in your credentials and then test it.  That’s it.  No muss, no fuss.  Take a look at the screenshots below to see the results.
Create your service
Create your service
Add your credentials
Add your credentials
SQL Server Get Schema
SQL Server Get Schema
We have now connected our SQL Server instance to our Microsoft Server 2012 R2 (both hosted on AWS) on my MacBook Pro.  Sometimes, it all falls into place.  Don’t forget to check out our wiki and community forums for more topics, information, and examples.
¹ I had my credentials on hand in a notepad text file for copy/paste quickness, but still, very fast 🙂

Filtering Related Columns within DreamFactory REST API Queries

Consider a query which joins employee records found in an employees table with information about their assigned department, the latter of which resides in a table named departments. The relationship is formalized using a key named emp_no. When DreamFactory parses the schema it will create aliases for each relationship, including one for the above-described named something like dept_emp_by_emp_no. The join query will therefore look like this:
/api/v2/mysql/_table/employees?related=dept_emp_by_emp_no
This would yield a JSON response containing records that look like this:
{
  "emp_no": 10001,
  "birth_date": "1953-09-02",
  "first_name": "Georgi",
  "last_name": "Facello",
  "gender": "M",
  "hire_date": "1986-06-26",
  "birth_year": "1953",
  "dept_emp_by_emp_no": [
    {
      "emp_no": 10001,
      "dept_no": "d005",
      "from_date": "1986-06-26",
      "to_date": "9999-01-01"
    }
  ]
},
If you wanted to limit the related fields to just dept_no and from_date, you would add dept_emp_by_emp_no.fields to the parameter list:
/api/v2/mysql/_table/employees?related=dept_emp_by_emp_no&dept_emp_by_emp_no.fields=dept_no,from_date
This query would yield records with the following structure:
{
  "emp_no": 10001,
  "birth_date": "1953-09-02",
  "first_name": "Georgi",
  "last_name": "Facello",
  "gender": "M",
  "hire_date": "1986-06-26",
  "birth_year": "1953",
  "dept_emp_by_emp_no": [
    {
      "dept_no": "d005",
      "from_date": "1986-06-26"
    }
  ]
},
You can learn more about working with related data inside DreamFactory on our wiki: http://wiki.dreamfactory.com/DreamFactory/Features/Database/Related_Data#Getting_the_Related_Data.

SQL DB REST APIs in Minutes, not Months

Would you like to access SQL data from your mobile, web or IOT apps?

If so, DreamFactory offers an easy and secure way to add a REST API to any SQL database in minutes, and supports 18 popular databases, among them MS SQL Server, Oracle, MySQL, IBM DB2, Postgres, SAP SQL Anywhere, SAP Hana, MemSQL and MongoDB! All you have to do is use the DreamFactory platform to connect your database, then use it to auto-generate a REST API for your database – it’s that simple!

In this blog post we’ll show how to REST-enable any SQL database, which is free forever for the databases and other services covered by our open source software. Then we’ll show some simple examples of how to use the REST API to manage your SQL schema and data.

Interested in a live demo with one of our engineers? We’ll be happy to show you how it’s done for your particular use case! If you’d rather watch a video, check out DreamFactory Academy. Alternatively, you might want to check out our new guide, Getting Started with DreamFactory. It offers a comprehensive walkthrough of generating a database-backed API. You’ll learn how to generate the API, define a role and API key, and then interact with the API using a variety of queries. Finally, if you haven’t already checked out our free open source software, you can download it here. Continue reading “SQL DB REST APIs in Minutes, not Months”