Around 1999 web designers around the world were struggling with the same problem: how to manage "large" web sites...particularly large catalogs of products for shopping cart web sites. Until then, many web sites consisted of either very long pages that listed many products, or one unique HTML page per item. It didn't take long to realize that this was going to be a maintenance and organizational nightmare. You have to put this into the context of other coding things that everyone was doing back then, like CGI-script shopping carts where even a non-technical designer could build a shopping cart by just changing the variables on a page by page basis to reflect different product pricing etc. All that seems so quaint now, although I'm sure there's a few folks still out there doing it this way!

Gradually we began to see ASP and Cold Fusion come onto the scene to provide a more structured approach to storing product and page data. As our freelancers started to pick up on the power of database-driven content, they began requesting hosting changes to accommodate early ASP sites. The first serious database driven site we were exposed to was Topbulb.com, which we inherited as a hosting and maintenance client. Back then it was pretty intimidating...very few web designers knew how to utilize databases for presenting web content on an HTML page, and ones that did were "well-compensated" to say the least. But like everything in web development, things gradually became more accessible. It was not strange to see databases handled with legacy languages or the likes of Microsoft Access or Foxpro because the few people that knew what they were doing were the folks that had been programming pre-web.

From almost the first moment on the scene, web developers realized that the power of database-driven content did not translate well to the search engine spiders (the roving programs that index the web for Google, Yahoo, MSN, etc.). At the beginning, it was not unusual for a spider to skip over most database-generated product pages unless they were explicitly linked from the home page (forgive us, this is getting a little foggy after all these years). If it wasn't for this problem, database-driven web site development might have taken over even more quickly. Those of us that were building large catalog e-commerce web sites at the time were scratching our head..."how do you get a client to decide between maintaining thousands of unique pages versus not getting any search engine rankings?"

Gradually search engines got smarter about indexing database driven content, but along the way developers came up with some very creative and also some very dastardly work-arounds to this problem, including modifying the URL structure as generated by IIS on the server to make each page look like a boring old HTML file, publishing HTML pages instead of dynamically generating them, linking all pages from master "site maps" etc. Since SEO rules the day, it cannot be overemphasized what a preoccupation this issue was! The seach engines have become much better, but it is still a problem that lingers in the background during discussions about navigation, site depth and coding style.

Once you get past the search engine optimization challenges, you quickly realize that storing product data in a database is a good idea not just for minimizing the literal number of pages on a web site...but for building a password-protected interface for modifying the data, the now ubiquitous "Admin". Since it is not generally a good idea to let novice programmers or designers mess with the code of a database-generated e-commerce web site page, it becomes essential to control the data on a day-to-day basis in some other manner. It could be said that the largest step "ever" in web development was the ability to have catalog web site customers go onto their own web site and modify their products, prices, images, categories etc.

Once you give a client a password to their web site interface and start expecting them to modify their own catalog data, it doesn't take very long for everyone to realize: "Hey, why can't I modify everything on the web site this same way?" The evolution of our web site products like Rapid Deployment and Proven Structure have emerged from this question. Once you have a client doing some of their day to day maintenance, you can't get them fully out of your hair until they can change their "Welcome page" "History" "Inquiry Form" and "Frequently Asked Questions" too.

Having witnessed the evolution of the use of database-driven web sites I now look back and say "how could we ever have thought that way?" about many of the approaches we took at the time and the "blinders on" view of the world we had. In today's web world, clients take about 15 minutes to get used to, and start taking for granted each new technology and ability web give them. Clients now know that web pages can display just about anything...in just about any style or manner...and they know that a a web site can be combined with a database to create a very tailored experience. Literally, you can store as many variables from a web site visitor as you want, and you can interrelate that data with server database data in any way you want. So now we have Ameritrade, online banking, product personalization, online registration systems, employment applications and calendar systems...all based on database-driven applications that run on web servers.

Chicago Web Design now has almost ten years of custom web application development experience with database-driven web sites. If we can help you make your project a reality, please contact us today.