I finally achieved a goal this week that I have been wanting to do with this legacy project for a while.
The ability to run single junit test withing Eclipse that does everything needed to execute an end to end web based functional test.
Architecture is
1 Spring AbstractDependencyInjectionSpringContextTests.
2 Spring BeanFactory containing embedded Jetty config.
3 Jetty is configured to point at 'exploded' war structure in Eclipse project
4. web.xml uses Spring's ContextLoaderListener to startup main Spring app context.
5. Selenium RC (server and java client) is loaded after App by another child BeanFactory xml file
6. JUnit test now proceeds, with full Selenium and access to all Spring beans in application context.

I am going show config files for this in a later post.

DbDeploy
I wanted to introduce DbDeploy to get the database integrated into our continuous build process. This is to avoid the bottleneck of pre-release sql-script gathering by the DBA.
It turns out that that there is an existing perl script that is used to execute all the sql scripts and has useful features like templates - useful for creating audit triggers from table column meta data.
So I stole a few ideas from DbDeploy and augmented the migration script to do these features.
The new features required were,

• Ensuring all sql scripts are under source control
• Storing a log of each sql script executed in a table in the DB
• Comparing source control with table log to only apply new scripts
• Detecting if previously executed sql scripts have been changed in source control and triggering complete restore from last nights prod backup.
• Failing cruise build on any sql errors

Now, with database continuous integration, app always works against db. No manual compilation of sql scripts is needed, all database changes comply with database integrity rules and naming conventions (cos of DB unit tests in build)

Following and adhering to database conventions.
Conventions in a database should actually be strict rules, and the db should be made to follow them. Unfortunately, there are many entities and many rules.
E.g. naming conventions, indexes, pks , foreign keys, trigger existence, privileges etc.

One way to help enforce this is for the DBA to police all scripts and hopefully spot anything thats not right.
The only way that works is to regularly run reports on this data that check for violations. Following an agile approach, I created a Java domain model to hold all database metadata and scripted all the database rules as JUnit tests.
Now, developers dont need to know conventions - they are taught them by the tests and the DBA doesnt need to manually check the db for any convention violations.

Continuous Integration
Having worked on systems with and without CI, its a no-brainer. An automated build system is needed.
I installed and set up Cruisecontrol to execute a build on every source control check-in.
The build compiles the app and then runs tests - that dont yet exist. Even without tests, knowing that the checked-in code compiles is an improvement...

Database migration (current production to next release)
Current process
The system has a new release every 3 months. Days before testing cycle, DBA must edit and clean up all database scripts created by developers over the previous 2 months.
DBA stays up late several nights in a row, continually getting a dump of production, restoring to dev database, editing scripts, running them against database and then re-editing until a smooth migration script is created that runs right through, with all missing FKs, and indexes added.
Problems with this

• DBA has no control over database changes because all the code has been written against it already, its too late to change the way its been coded.

• Developers make bad database design decisions
• DBA might make changes to the scripts that break the Java code
• It takes up all the DBAś time for a couple of weeks and stops whole project as it is a bottleneck.
• Developerś changes may not be compatible with each other.

Proposed Solution
Database Continuous integration - e.g. DBDeploy

Environment management
There has been a massive increase in the number of environments from 2 dev,1 uat and 1 prod
to about 9 UAT environments 3 dev and one prod.
This has caused a lot of work, enough for a new hire just to take care of them all.
Issue from Java side is, all the environment specific settings were in properties files inside the deployed binary.
We could then just select the internal property file using a -D java command line parameter.
This has proved not to be scalable
1. Settings change much more frequently and every change requires a new binary build.
2. Source code branching of the app also means we get multiple versions of the same environment file - very confusing..
The solution I think is to move environment settings to separate source controlled project. This then becomes like a environment directory, always of current state.
Using JNDI is another solution, but we also need these environment files to be processed by unix scripts.

Too many tools
A lead BA let us know the new 'requirements gathering' process today, basically a business workflow process. Regardless of what this was, the following is true (in my case).

Development work must be prioritised to deliver the most important thing for the business. Development work is triggered from
1. User comes up with a new requirement that will make their life easier
2. BA comes up with a new requirement from analysis that will improve efficiency/cost
3. A bug is found in production that needs fixing
4. A bug is found in testing process that needs fixing.

Various people need to be aware of requirements and track them to completion.
I.e. Users, BAs, project managers, developers, testers etc.

What was madness was the solution used a different tool to log each of the aforementioned items. A helpdesk tool, a user idea request tool, a testing tool and a separate requirements gathering tool.
All the actual development would then be done against another tool e.g. Bugzilla type thing linked to source control.
Any hope of tracking anything without double or triple keying is going to be tricky.
I haven't used it myself but I think JIRA would be worth a go to do the lot.
I have used TRAC and that was ok after some customization