The history of software development is filled with failures. A 1992 survey/study indicated that > 80% of large software projects were failures: nothing was delivered, or was delivered > 1 year after the target date, or was missing major functionality. A follow up study in 2000 did not show much improvement. The lesson of software history is clear: don't do large projects!
In the last decade a new software development methodology has arisen: Agile Project Management. The core concept is: break large projects into a sequence of small projects. Only commit to doing what one can reasonably plan and estimate. Deliver each mini-project completely before starting the next. Review progress to date, adjust plans and schedules, then start the next mini-project. Much success has been had with this approach.
We use a form of Agile Project Management known as Scrum. The methodology is composed of the following steps:
Step 1. Collect the initial requirements and functionality as far as can be determined. Often much is murky and unknown about the ultimate target, but we can start with what we do know.
Step 2. Develop a prioritized list of tasks that implement the known requirements and functionality. Each task should be limited to a few hours duration.
Step 3. Select a set of high priority tasks to implement within a short development cycle, usually one to two weeks. The set of tasks chosen must result in a deliverable, tested, working version of the target application.
Step 4. Implement, deliver and demonstrate the current version of the application to the client/customer.
Step 5. Review the requirements/functionality for changes based on the just delivered version of the application and/or changes in the external world. Modify the priority task list to reflect results of the review.
Iterate over Steps 3-5 until project is complete or client/customer cancels project.
At the end of each cycle the client/customer has the opportunity to revise, redirect or cancel the project.
At the end of each cycle the client/customer has a working version of the application.
Success is achieved each and every cycle, one cycle at a time.