Representational State Transfer (REST) has been around for more than a decade and continues to grow in popularity. There are dozens of frameworks that can expedite implementing a solution; 16 OpenSource PHP alone. So when Wayfair Engineering decided to evaluate migrating to Web Services (WS), it was a no-brainer. REST, right? Wait; there are other more mature frameworks.Other leading candidates were SOAP and WSDL. Both were quickly dismissed due to Security concerns (RPCs through HTTP ports), being top heavy, and server-side complexity. SOAP is great at exposing interfaces to PHP, PERL, and other languages but lacks simple HTTP APIs. Calls to a HTTP API should be as easy as cURL or as robust as a Thin-Client. For us, some non-factors were bandwidth, rigidity, and existing support/tools.

Let’s not begin with Episode IV!

First, our high level design and framework considerations were;

• Latency incurred by additional processing should be negligible
• Scalability
• Ability to separate data, business logic, and presentation
• Support multiple response formats: JSON, XML, and POT (Plain Old Text)
• Ease of code migration. I use the term “ease” loosely. We have Customer Support teams writing SQL. It is great to support open data but designing a comparable system can be a challenge.
• More diligent about exposing functionality and data (a best practice)

Truth-be-told, we did not fully adopt REST. In the eCommerce ecosystem, there is not always a simple way to represent an object as a distinct URL. Sure, the VERB usage maps nicely but there are cases where the NOUN/Adjectives can be very complex.

• Simple case – Retrieve information for a given order
• Complex case – Retrieve information for B2B Orders in June over $2000

This illustrates where we decided to diverge from the traditional REST conventions. As you can see [Complex case], the URL includes a query string with optional parameters. Query string parsing is available via libraries but the validation and consumption falls squarely on the controllers. As a team, we opted to base Validators and Extensions in the framework but wanted to provide flexibility at the controller level.

A simple compromise was made with the response format. Undeniably, JSON has a very simple data structure that can be parsed by most languages. For 90% of the use cases, this is sufficient. But, not to belabor the point, the Complex case demonstrates the need for more structure in responses. For clients that need more control, XML schemas can be defined and data transformed with XSLT. Some of the arguments made to abandon SOAP can be applied to XML. Nevertheless, there are benefits to both. Fortunately for us implementing a response in XML does not require much effort. Our framework, built from the ground up in less than two weeks, is completely modular and extendible. With little effort, features can be tacked onto the framework and controller in parallel; taking advantage of the version scheme maintained in the request.

To the code:

My original vote was to write everything in C++. Sadly, I was overruled. PHP was selected as the primary language for a number of reasons: widely used at Wayfair, OO, and, did I mention, there are 16 REST Frameworks to choose from. The Routing Engine is far simpler than it sounds. It matches parts of the URL to specific controllers. Controllers register with the engine, today this is static but require little engineering effort to make dynamic. The Super Controller class is an abstract class forcing each “working” controller to implement required functions. Some of the other smaller pieces are the request and response objects, a dispatcher, and emitters.

Topic for another post; Episode V: The Emitters Signal Back. That was a stretch….

On the roadmap:

More long-term we will be expanding features such as caching and status messaging.

• Caching. The nature of the GETs coming from clients allows us to cache a higher percentage of requests.  Admittedly, there is a class of result sets with short TTLs; similar to the Complex example above.
• Security – single sign on (CoSign and Pubcookie), DMZ proxy servers

 

The best part:

Our framework was so well designed and implemented that all of the clients are automatically generated. These clients come with “batteries included” (Thanks Gavin) meaning the Factory methods can be utilized with a couple lines of application code. Extending support for additional languages required less than a day’s worth of development team. Originally written in PHP, we added ASP and JS in half a day.

Another “advance” feature is the routing engine that interfaces with the web server. It became very clear early on that we did not want to keep web server condition/rewrite rules in the web server config file. Adding new controllers would require a restart; not optimal. The routing engine is based on the same principle but done by the pre-controller part of the framework.

In summary:

Initial impressions indicate that this was a highly successful endeavor. There is now a clear distinction between data, application, and UI. The traditional Wayfair Software Engineer still must be a well-versed in T-SQL/CLR, PHP, and HTML/JS but the shackles of code-data dependencies have been removed. Wider adoption is our next goal.

If this type of thing interests you, please visit our careers page, and join the team!