This article first appeared in issue 223 of .net magazine – the world's best-selling magazine for web designers and developers.
The killer app is not just about coming up with a great concept and a great design. If you’re not ready to become a real operation you may fall at the first hurdle. But with so many technologies, products and development styles available, where do you start? Make careful decisions right at the beginning and growing up won’t be as painful as it could be.
Designing, delivering, maintaining, and how to build an app all rely on good technology selection and software architecture. Important decisions need to be made early on across a number of dimensions: speed of development, maintenance, operations and the ability to scale and grow cost-effectively and quickly. Once the fundamentals are in place, the app is in a good place to grow. Time is freed up to focus on the product and its features, allowing you to explore new directions.
Don’t waste your time trying to solve problems people have solved before. Take out the heavy lifting by researching a number of areas early on:
- Programming languages (your skills, toolkits you want to use)
- Frameworks (‘scaffolding’ code to help boilerplate the production of different parts of your app)
- Libraries (pre-built code that someone has already written and proven that you can reuse
- Software architectural style (approaches to integrating different things, API exposure)
Separation of concerns
Separation of concerns is good practice in software development: it’s about sub-dividing the functionality of the code so it doesn’t overlap. Different parts of your app will most likely be written in different programming languages, and by people with different skillsets. The modular pieces of an app will have different characteristics – some code might not change very much (such as the core product engine; “your magic algorithm”) while other parts might change on a regular basis, such as the text that appears on web pages.
Separate groups of developers should be able to work independently on different parts of the app’s software landscape. Parallelism of development means you can get to a build drop quicker. Decoupling the different technologies means developers won’t be falling over themselves on the same version control artefact, causing a bottleneck and reducing speed to market.
Dynamic HTML content generation – web pages with user-specific data in them – is achieved through the CGI programming model. The HTTP web server that’s interacting with your browser can execute “CGI scripts”. The programs write their output back to the web browser as text in HTML page markup.
In the early days of the web, the most common scripting language was Perl, or compiled programs, such as in C. Here’s some oldschool Perl.
Call acme.com/example.pl to invoke#!/usr/bin/perl# example.pl# business logic / user data$message='.NET magazine rocks!';# begin HTML mark-upprint "Content-type: text/html\n\n";print "<html><head>\n";print "<title>CGI Test</title>\n";print "</head>\n";print "<body>\n";# embed user dataprint "Today this site thinks $message";# finish mark-upprint "</body>\n";print "</html>\n";Today the modern equivalent is PHP with an extensive set of libraries. Plus there’s the complexity of Ajax and the ability for browser code to interact with the HTTP server and manipulate the user interface.
In this example, printing the message of the day is the ‘secret sauce’ logic that makes up the site. The code for markup and business logic is mixed together. If you need to make both a visual change to, say, the layout of a table, and a logic change to get the data to populate the table, both developers need to code and test in the same script. They’ll need to spend time working out how to bring their changes together (Figure 1).
Now think of the site as hundreds of pages built and maintained by a team of frontend developers, building the user interface and experience, backend developers creating the logic and data processing that powers the site, plus infrastructure developers, designing and building databases and connectivity to external worlds. To live up to that v2.0 expectation you face some challenges bringing these different parts together.
Web development frameworks help you implement the Model-View-Controller pattern and achieve separation of concerns. They separate the server logic and HTML markup into two different code artefacts and provide a specification on how they intersect; the framework code runs as part of the server and manages the overall request/response lifecycle (Figure 2).
If you’re using Java at the backend, Apache Wicket is currently one of the cleanest HTML and logic separations; other frameworks include Apache Tapestry and the Apache Struts.
All these frameworks will be useful in helping you achieve separation of development concerns. All of them have overlapping features and pros and cons. Your choice will depend on the backend language you want to use. How the framework handles scale and performance should figure into your research.
Get the Creative Bloq Newsletter
Daily design news, reviews, how-tos and more, as picked by the editors.
The frontend
Frontend code is all about the user. It needs to present content and visuals, allow users to interact with the content and dynamically manipulate information, and invoke transactions.
The term “web app” has become stretched these days, and a number of choices face you in terms of technologies. Do you want to build on open standards and avoid proprietary technologies such as Flash? Do you want it to work with mobile (almost certainly yes)? Do you want a native iOS and Android app to take advantage of device-specific features, or to use standards-based HTML? Do you have to live with a legacy set of users with older browsers, or is your app so amazing people will change browser just to use it?
It’s a no-brainer to start with separate HTML and CSS files. There are many approaches to well-organised CSS development: consider the use of multiple CSS files for different aspects of style, separating common styling versus browser, device or international styling. Recently a number of CSS frameworks have emerged to make page layout easier.
JavaScript has risen to become a powerful technology to enhance features and functionality. With it has come a number of frameworks, toolkits and libraries: pre-built proven code that can help with much of the UI development heavy-lifting. There are also popular UI-rich libraries such as jQuery UI, Dojo and Mootools.
The backend
The backend code is your app’s engine room. While the frontend focuses on user interaction, the backend is focused on data processing. It stores and retrieves data, and enables transactions, for example, taking payments.
You may need several technologies, usually centred around a core programming language and suitable framework. Frameworks do two things: implement the heavy lifting code of features you shouldn’t have to design and build, such as internationalisation and logging, and organise the code design into a few simple patterns. This helps you achieve repeatability – coding faster as your familiarity increases – and simplify maintenance.
All sorts of languages are available, depending on your skills and what sort of features you’re looking for. The ‘traditional’ web app languages (PHP, Java, .NET, Python) come with innumerable mature frameworks and libraries. Some of these languages are, however, showing their age – their development processes can be cumbersome.
Picking a backend language depends on your integration style. The language needs to support client libraries that can talk to your external partners, such as the need to support RESTful client HTTP calls.
In recent years there’s been an evolution from Java or .NET language platforms to those such as Ruby on Rails and Grails. They embody “coding by convention” to simplify the development process.
Integration strategy
By now you should be seeing the beginnings of your software architecture. It’s tempting to start with the HTML concept and grow out from there. It’s time to think about the app as a multi-channel platform.
HTML and JavaScript in the browser is very powerful. You could, for example, use browser JavaScript code to connect to Facebook and get information about your friends to be represented in your page (see Figure 3).
But loading up all that code into the browser can restrict growth. What if you want to augment and manipulate that Facebook data with your app’s data? That might be your secret algorithm; you don’t want that code visible in the browser. What if you want to take advantage of the features of a particular smartphone platform and build a native app? You would need to do that same integration and data manipulation for presenting in the native app. You’d end up with copies of the same logic, in different languages – another maintenance headache.
A better approach is to integrate in as few places as possible. Isolating this processing can help with scaling (perhaps on a specific server) as well the obvious maintenance factor (see Figure 4).
Build a platform
Apps are no longer expected to be a single platform. The success of Twitter hasn’t come from its website but from the platform built around its API. Twitter clients and platforms built by others have sprung up, hooked into its exposed services.
The ability of a platform to be integrated with other platforms for other purposes is in itself a market opportunity. The ability to integrate your services at a data level opens up reciprocal opportunities with other platform providers. The more ways you can offer integration, the more ways there are to find opportunities.
Services such as LinkedIn, Twitter and Facebook have multiple technological end points that invoke the same service (‘the API’). The access might be a lightweight RESTful approach or require more heavyweight SOAP services (especially if, for example, secure, non-repudiation interactions are required using digital certificates). The data structures could vary from JSON based to XML-schema driven representations.
Databases and State
The heart of your service is its data. The traditional approach is to use a relational database management system (RDBMS). Entity-relationship modelling of database tables and SQL data manipulation is a subject in itself. The intent of an RDBMS is to provide programmer intuitive interaction with data, and hide how the data is stored and maintained (the purview of the database administrators).
The most popular open source database is MySQL (mysql.com). Many of the features of ‘enterprise strength’ rivals, such as IBM DB2, Microsoft SQL Server and Oracle Database, are making their way into MySQL, and it’s rapidly becoming the de-facto RDBMS for web apps. Facebook is its most testing of users, while Twitter’s engineers give an insight on how they built software architecture for search around MySQL here.
Not all your data need be processed and maintained in an RDBMS. In fact, if you have lots of data objects associated with the user, such as large files, PDF rendered copies of data and binary objects – such as photos and other media files – then you could stretch its capability and the hardware cost may be excessive.
There are new approaches to data processing for web applications. Where you can be less rigid on the data structures (and their relationships) there are other solutions to help with large-scale storage and scaling. Consider NoSQL databases such as Apache Cassandra, a key-value store database, or Apache CouchDB, a document-oriented store. NoSQL DBs are good for data-intensive needs, such as serving pages on high traffic sites or streaming media.
When you need raw storage of things such as photo and media objects, and you want to make good use of server resources then consider high performance distributed file system technologies. Hadoop is more than just a distributed file system, but it was inspired by the Google File System focused on write-little, read-often of large media objects such as video clips. Hadoop offers MapReduce processing for processing large sets of data if this is important to your app’s functionality.
With any database approach you need to consider the programming technology you’ll need to invoke the storage and retrieval of data. The data needs to be transformed so it’s suitable for rendering in a web page. That processing could be close to the web page (JavaScript/JSON) or the server logic (PHP, and so on). The established RDBMS world will have frameworks to help, the newer NoSQL world might require more programming.
New technologies
It’s worth mentioning some of the newer tech on the horizon. We’ve already mentioned the rising use of NoSQL databases and distributed file systems. Node.js is the next generation of high throughput web server implementation. Exposed as server-side JavaScript it offers a modular approach to building network scalable apps – serving many more processing connections for users over traditional Apache or Tomcat-based servers on the same hardware. It’s early days for node.js; other frameworks, such as Netty, come with a variety of protocol and integration libraries.
Functional programming languages such as Erlang (and hybrids such as Scala) are being used more for the backend processing. It’s argued that these types of languages are better at processing on multi-core server architectures and will have more advantages in cloud deployments. Twitter found parts of its site were stretching its standard Ruby on Rails infrastructure and switched to a Scala-based system.
Other backend core languages you could investigate include Google’s Go as a core system programming language, and X10, designed for parallel programming.
Conclusion
There are enough market forces and unknown growth opportunities to complicate your startup. So you need to create a simple plan of action for all the different technological releases you will need.
Think about the design of your service as a platform – create a software architecture that can support that API access when you come to build it. Think about screen input and design separation so you can easily build for multiple devices. Think about how to process the data that powers your app and how it’s going to be able to scale massively.
Discover the best iPad apps for designers at our sister site, Creative Bloq.
