Second-System Syndrome

I recently had the opportunity to help out a friend and did a quick spec for a dynamic web application, I’ve been doing some front-end mockups using Kickstart and Chart.js and Font Awesome. It has been fun, but the funny thing for me about it all is, the general “technology” that the web app represents, is one that I have fairly extensive experience with, Mutli-Tiered Product Distribution Portals (best “technology” name I could quickly coin).

I have a general philosophy on how dynamic web applications should be architected, not to say that it is a rigid position, one should always strive to use the best technology and architecture/patten for the goal at hand, but for me a general web application should ALWAYS have a Data API Layer.

It is an investment to go with a Data API Layer, but one that tends to pay off with big dividends and I feel that the web industry as a whole has latched onto this concept, but definitely smaller shops, they just want to write their code to access the database directly, to get it done. Which I can appreciate, but it hurts my professional soul.

The Data API Layers I’ve written in the past exposed business logic that was critical to the success of the product to partners that would have otherwise had no ability to really integrate the selling of my client’s products. Without that Data API Layer, that client would have been forced to have their partners integrate directly with their vendor. And then the client would have had to try and cobble together a billing system from the vendor’s reports and spreadsheets and accounting minions, or try to frankenstein’s monster a more automated system that would passive poll the vendor system for data.

None of it IMHO works well for a real-time product sales/distribution system. It also creates vendor lock-in that every business generally wants to avoid. So with the Data API Layer I created for them, they were able to have a self-contained billing system that allowed for easy export to quickbooks, locked the partners (to a degree) into the my client and allowed my client to switch vendors (without the partners needing to retool anything with their integration).

With all that said, there was more I wanted to do, things I wanted to build in. The Data API Layer I built was/is very robust, but very spartan in terms of metrics on usage and things one might expect from a more productized API, which it wasn’t. Furthermore, the Multi-Tiered Product Distribution Portal system had features we added over years of experience working with sales people in the field, but still there were features and such that I would put in, particularly knowing what I know now, vs dealing with tables that have thousands and even millions upon millions of rows of data.

Enter what some call ‘Second-System Syndrome’ or the ‘Second-System Effect’ .

The quick summary of this concept is that, a designer/developer attempts to build in everything they wanted to do with the first version of the system, but couldn’t due to time constraints and such.

Yeah, my quick spec for my buddy, I don’t feel was that at all, I feel it more along the lines of a refinement of a system I have deep working knowledge of, but with the benefit of not having to worry about data migration, furthermore, the first system was all about Virtual Product delivery, this one is dealing with physical products.

So right there, a number of issues come up that are not there in the Virtual Product world, you know like inventory, shipments, replenishment, etc.

Due to my strong belief in Data API Layers, many of my personal side-projects have been in sort of a holding pattern, having a newborn child also tends to put personal side-projects on hold as well, but I lacked an API engine to allow me to really get started. Because to me, just writing the code to directly access the database, is being lazy.

Before my friend contacted me about needing the spec, I had already been working with Phalcon (which I plan to write a few articles on my experience in an effort to help others, I love Phalcon for API work, but their documentation makes using their stuff very difficult), and building an API layer for my own projects. I had just gotten the first version of it working and was starting to build in the more useful features that I always wished my other API engines had or taking inspiration from an API engine that I worked with years and years ago, that was so dynamic and database driven that well it was amazing, but it was also daunting to configure and had that nasty habit of taxing the database engine (memcache would have likely fixed that easy).

Point being, using my fairly vast professional experience of consuming, designing and developing web services, to build an API engine that I feel could be core of multiple dynamic web apps and yet not so universal that it is trying to do everything for everyone, because “To please everyone, is to please no one.”

It will be interesting to see what becomes of the spec and mockups I put together for my friend.  I don’t know if I would go with Kickstart for a production project, it definitely made quick mockups easier, but I feel that I would want to go with a more ‘complete’ solution like Bootstrap for CSS, Chart.js feels pretty good (especially for free/open-source) and Font Awesome is perfect for all the needs I can foresee.

In case one is curious, I am favoring Laravel for the MVC server-side scripting (PHP) framework for my side-projects with the API Layer being written in Phalcon (also PHP server-side scripting framework). Which brings up another topic that I will need to defer to another post, but the title would be something like “Why use a framework?” There are plenty of reasons not to use them, but for me there are so many more on why one should use them. I’ll probably write that one next.