Will Agile Development Bring Back Custom Software?

Ann All

The other day I was griping about our content management system, an activity I and other editors indulge in fairly regularly, as do members of our small IT staff, who probably get sick of telling us that it'll take a fair amount of custom development to add any kind of new feature to our site. In fact, although the system is not yet 2 years old, there is a growing chorus of folks lobbying to replace it.


Perhaps the system wasn't vetted as thoroughly as it should have been. I don't participate in technology evaluations, so I wouldn't know. But I think the problem is more fundamental than that. As I told my boss the other day, content management is one of those areas that, for companies like ours, is filled with a surprising number of specialized processes. Thus any off-the-shelf software application -- not just the one we are currently using -- would likely fall short of the functionality desired by our users.


Less than a week after having this discussion, I saw a Dr. Dobb's Journal article written by Mike Jones, VP of software development company Outsystems, that summed up my feelings about packaged software pretty well. He wrote:

Best practices aren't necessarily one-size-fits-all, especially in corporate IT. What works for one IT environment may burn another to the ground. This is where off-the-shelf software falls short -- most of it might work with your enterprise, but parts of it will need more than just "tweaking" to work with your business. With large amounts of customization, package implementations become very brittle and expensive, often resulting in difficult upgrades and an inability to respond to changing business needs.

Yet despite this reality, companies have long bought into the idea that they can shorten development cycles and cut costs by starting with packaged applications. That was Waste Management's goal when it purchased ERP software from SAP, according to a CIO.com story recapping the company's two-year lawsuit, $100 million lawsuit against SAP, which ended earlier this week with a one-time, undisclosed payment from SAP to Waste Management.


Whether it's packaged or custom-built software, the crux of the issue might be users who simply don't know what they want or have trouble conveying it to developers. In statements responding to Waste Management's complaint, SAP faulted Waste Management for not adequately defining its business requirements. Jones writes that "business users typically don't know exactly what they need from an application," which leads to lots of rework and results in time and budget overruns.


To be fair to users, it's not so much that we don't know what we want now; but it's hard to predict what we'll want months down the road given the rapid pace of business change. Jones, whose company specializes in agile development, not surprisingly promotes it as a solution to this problem. He writes:

With Agile, developers start with a high-level set of business needs to guide their team. The team then addresses the most glaring or important need, and shows a working demo to the business users within a very short period of time. User feedback is solicited to add more detail, and then iteratively and continually to build out the system. By embracing and expecting change upfront in the project, IT and the business can work together, building the right solution based on business feedback and priority.

Sounds good, right? Agile appears to be gaining momentum, with a recent Forrester Research survey finding up to 46 percent of IT pros said agile software development was being used at their companies. It's far from a panacea, however. A lack of communication between business users and developers will derail agile development projects just as it will wreck more traditional waterfall development projects.


We use agile development techniques here at IT Business Edge. When I interviewed Steve Hardin, ITBE's VP of software development, about agile, he told me it's easier to get both developers and business users "excited and focused" with agile, with its emphasis on breaking a software project down into smaller increments and producing more frequent deliverables. However, scope creep can be an issue, thanks to agile's at least theoretical flexibility in adding new features. If this occurs, said Steve, "you have to go back to the original business need" and make clear it'll require extra time and money to add features.

Add Comment      Leave a comment on this blog post
May 4, 2010 5:58 AM Paulo Rosado Paulo Rosado  says:

Ann, I think you isolated the fundamental problem with any App Dev methodology: the cost of software change (or feature creep). While Agile increases alignement between business folks and IT the fact is that if the cost of change is still going to be 50x times more expensive after you have a deployed version of your software, you will eventually slide into waterfall as your sprints become increasingly more difficult to deliver. This is why we are seeing more frameworks and platforms that concentrate on decreasing the cost of change. This is done by automating deployment, automating performance management, helping track software dependencies and in general support developer knowledge transfer.   

May 5, 2010 3:12 AM Mike Jones (who) Mike Jones (who)  says: in response to Ann All

I can't resist - this is OMJ. "Original Mike Jones" and author of the referenced article. If you are wondering, I rarely get confused in person with my rapper name sake. 

Ann makes a very important point.  They are using a hybird Agile approach to meet their development needs. For most corporate IT Dev this is critical for scale and success. I like to call it "enterprise agile". Check out the blog www.aboutagility.com for more. 

May 5, 2010 9:16 AM Al Gibson Al Gibson  says:

Of course the guy running the Agile Development company is going to say that Agile is the way to go.  That's kind of like asking a bankruptcy attorney if bankruptcy might be a good option even if you only have a tiny amount of debt.

Who is Mike Jones?  Is he still tippin' on four fours?

May 5, 2010 9:42 AM Mark M. Mark M.  says: in response to Al Gibson

@Al - you missed Ann's point. She was talking about agile development in the context of ITBE's own experience, about how much custom development they'd need to do to their content management system to add new features. Its an issue most of us I'm sure can relate to. She is largely talking from an internal perspective about their own experiences -positive and negative.

You do get points for the Mike Jones rapper reference though.

May 5, 2010 10:44 AM R. Lawson R. Lawson  says:

Agile, or any other methodology, probably won't have a major impact on these decisions.  And I agree with Forrester: "A lack of communication between business users and developers will derail agile development projects just as it will wreck more traditional waterfall development projects."

My problem with Agile isn't that it's not a viable methodology.  My problem is that people see it as a silver bullet that will solve all of their problems.  It's become a trend and as with any trend people follow suit simply because everyone else is.  It's to IT what diet pills were to Anna Nicole Smith.

I think the days of $100M outsourcing deals should be over, but I suspect they aren't.

May 5, 2010 11:01 AM Ann All Ann All  says: in response to R. Lawson

The sentiment about a lack of communication is mine, not Forrester's. (Though I'm thrilled it sounded intelligent enough for you to think it came from Forrester) I cited Forrester's study, which indicates a growing popularity for agile. My concern: While many organizations introduce agile for the right reasons (desire to speed development times, produce apps more in line w/ user expectations, etc), others may do it because "all of the cool organizations are using it." Paulo is right that more frameworks and platforms are being introduced that can help decrease the cost of change, sometimes by imposing more needed structure on the development process. Here at ITBE, we actually use a development process that combines agile w/ some aspects associated w/ waterfall development.

May 7, 2010 2:51 AM Derek Cheng Derek Cheng  says: in response to Ann All

Ann, I have to agree with you. Agile on PaaS (Platform-as-a-Service) implementations actually reduces cost significantly, because most "features" are modifications of process, reporting, or to the data model which can all be done instantly and even handled by users themselves in a PaaS environment. In addition, PaaS enables Agilers to focus on core stuff rather than dealing with commonalities in application development.

May 11, 2010 11:57 AM Ray George Ray George  says:

Ann - I'd be curious to hear from Steve Hardin, ITBE's VP of software development, your testing best practices (both unit and functional) as part of ITBE's development lifecycle.

May 12, 2010 11:22 AM Steve Hardin Steve Hardin  says: in response to Ray George

The first step for us is to ensure the development team has the experience to understand the end user requirements through user stories.  When we do this right, testing becomes integral to the development process rather than something you bolt-on at delivery.  We enforce accountability for unit and functional testing on the development team.  However, we do not mandate Test Driven Development (developers write failing test case and then write the minimal code to fix the test).  Instead, we allow developers to use what ever process they find most efficient to ensure the software meets the end-user requirements.  We make this easier, by delivering frequently and in small increments that are easy to understand and generally self-contained.

Some of our developers who find hardcore Test Driven Development an effective use of their time, and we support them.  As I mentioned, we don't mandate TDD:  sometimes the project scope doesn't require it or sometimes you have a developer who writes more efficiently with very low error rates without writing very specific unit tests.

As tools improve, we are to implementing Behavior Driven Development where the end user stories are captured in code.  For us, this is a more efficient mechanism for ensuring quality software at delivery to QA than having developers write 100s or 1,000s of TDD unit tests that may not be immediately relevant to the end-user requirements.  Our test team is aware and involved in the user requirements from the beginning and are encouraged to understand the end-user requirements as the software is being developed so that when software is delivered to QA, that they understand what needs to be tested.  We don't try to test in quality and our testers don't play a "gotcha" game with the development team.  Ultimately, the development team is accountable and becomes the customer of the test team and end-user as software moves to delivery.

May 24, 2010 1:13 AM Vinod Narayan Vinod Narayan  says: in response to Steve Hardin

Another important part when you do agile development is to also pull in a common process within the traditional method in a slightly different manner.

And this would be understanding Business Impacts. At the end of the day we all relay tremendously on an individual's ability to understand and deliver and in that case, while increments help in delivering better, it will be a good idea for the team members to be aware of the business impacts associated with the end of each sprint.

By Business Impacts I don't just mean the impacts it has on the customer or end user, but also the impacts it has on the product/project owner with respect to his managing the expectations of his managers or at the most the expectations of the person or group funding the project.

From my personal experience I have been able to get better output from my team members once I have been able to explain the exact impact a sprint has in the overall project.

I would recommend these as two different strain of activities, one that follows the process to deliver and the other that follows a process to keep people updated on changing impacts in a form in which every one feels aligned to a goal and not get short sighted by a sprint to sprint approach

What I have done and keep doing is introduce an element called market readiness into the whole equation that brings the realities and dynamics of the streets on to the white board



Sep 13, 2010 11:25 AM Custom Software Development Custom Software Development  says: in response to Vinod Narayan

Ann, I totally Agree with you but i have some point is most important in custom Custom Software if any development team first understand user or client requirement as define user and another important point Development team also give full facility to user and give more benefit to user so user can understand better way.thanks for provide this information and i hope you give me more information in future.

Dec 20, 2010 2:44 AM SaaS SaaS  says:

Clients not knowing what they want and adding to the scope of the work is a big problem.  And the developer is the one that gets blamed.  Even when the scope is clear, the client can add things and then get angry at an attempt to add money for the addition to the scope.  This is very frustrating when you work so hard at something and there is no understanding of how much work it takes.  How can we communicate better with clients?

Aug 8, 2011 8:12 AM Nick Nick  says:

Nice job you have done on this blog.....

Around You iPhone Application : This app allows you to find out information 'Around You' on most of the common needs like ATM, Banks, Gas Station, Hotels, Disco, Theater & Hospital Etc...

Sep 11, 2012 12:15 AM thomas thomas  says:
Brilliant list, has made sourcing good content for reading and blogging much better, also some great inspiration in there. Reply
Nov 1, 2012 11:58 PM Harding Sargent Harding Sargent  says:
Yes, nowadays bespoke development market are going very highly.and all are want to concentrate only on bespoke software development. Reply
Jun 24, 2013 1:50 AM ChadwickYates ChadwickYates  says:
Bespoke application growth is otherwise known as customized application growth. Remember that every business is exclusive and thus it needs personalized application that fits its exclusive needs. Reply
Aug 12, 2013 4:22 AM jacob jacob  says:
Custom application development is far better than using already built software package because it allows you to get a software solution that is developed according to the unique business process of our business. It reduces the need of changing the process of business according to the packaged software because the software itself maps our unique requirements. Reply

Post a comment





(Maximum characters: 1200). You have 1200 characters left.



Subscribe to our Newsletters

Sign up now and get the best business technology insights direct to your inbox.