Extreme Programming for Small Scientists?

We have a new autumn intake of researchers into our Centre and are aware that there are constantly changing demands on the software and informatics skills needed. In Big Science projects there is provision for infrastructure and training and well-developed methodology for the creation of software. We're working out what is appropriate for "smaller" sciences like chemistry.
"Small" is not inferior – in fact it can have advantages, allowing faster and more diverse activity. But there is less formal support and software usually has to be done in the margins of projects. Software per se has little positive formal reward in science as it is the research in citable papers that matters to the evaluators, regardless of the value to the community.
So how do we develop a good, modern, software environment and lead people to best practices? Today I'll start with Extreme Programming (XP) which talks a lot of sense (I quote from Wikipedia):

Extreme Programming Explained describes Extreme Programming as being:

• An attempt to reconcile humanity and productivity
• A mechanism for social change
• A path to improvement
• A style of development
• A software development discipline

and

… five values are:

• Communication
• Simplicity
• Feedback
• Courage
• Respect (the latest value)

and summed up in 12 practices, grouped into four areas, derived from the best practices of software engineering:

Fine scale feedback

• Pair Programming
• Planning Game
• Test Driven Development
• Whole Team

Continuous process

• Continuous Integration
• Design Improvement
• Small Releases

Shared understanding

• Coding Standards
• Collective Code Ownership
• Simple Design
• System Metaphor

Programmer welfare

• Sustainable Pace

Now, XP is aimed at teams of developers in commercial organisations creating saleable products against whose success the team can be measured. Whereas small scientists are often working singly on projects with no positive software metric. So can XP (and it has many critics) be relevant?
I think some of it can. The five values are de facto attributes of a successful Open Source project, so by adopting Open Source (especially on a distributed global model) you have to adopt these. You cannot grow a successful group if they do not communicate, write simple code, give feedback (bugs), have extreme courage (more than XP demands), and have respect. So if we can translate the Open Source values into local practice then we have imported these values, regardless of the size of the projects. Of course there has to be some shared goal, but most research departments probably provide some of that (there will, of course, be some individuals working in such new areas that they have no natural companions).
Of the 12 practices some are only applicable to commercial and quai-commercial organisations (perhaps in Big Science). So my list is something like:

• Pair Programming. Where possible someone else should work alongside you some of the time ("mentoring" could be a better word). The second person need not be an expert programmer, but may be good at designing information or act as a rubber duck.
• Test Driven Development. Absolutely essential. Junit tests (more in later posts) have revolutionised my programming – I couldn't libe without them.
• Whole Team. Not easy, as not everyone belongs to the same team, but valuable if possible.
• Continuous Integration. Yes. Things change so fast that we cannot work with infrequent large releases. Working on Sourceforge we are used to nightly builds and welcome them. Of course the nightly builds have to pass the Junit tests!
• Small Releases. Again with the Sourceforge mentality this is standard practice. It does require careful attention to APIs – too many changes and the re-users get disillusioned. For example I decided to refactor the namespace for CML (there were just too many variants) to a single namespace for all time. One of my valued users told me that he would just about tolerate this, but any more and I was dead meat!
• Coding Standards. Difficult to enforce socially, but happily the tools (at least in Java) implicitly set standards. Tools such as PMD are very useful and we can hopefully standardise on a set of style guides which are not too picky
• Collective Code Ownership. Again any Sourceforger gets used to this. But it can be more difficult within a real-world group.
• Simple Design. Fundamental, but not easy to learn or teach. Like architecture. You know it when you see it! So emulation is a good approach and constant code review by others. The balance between YAGNI and anticipation of requirements is difficult.

I add to this things that seem obvious to the commercial developer but by no means so natural to the Small Scientist

• Use an integrated development environment (IDE).  These are now Open and very impressive. We use Eclipse for Java and are going to recommend it to everyon in the Centre.
• Standardise on libraries. Again that is difficult in some cases, but we can now do this with Blue Obelisk for chemical informatics. After all, some of us have spent enough time developing it!
• Present software projects to the assembled group even if they are on different projects. Be honest – what went wrong is often more valuable than what went right.

So – if you have read this far – we would be very grateful for any feedback from other Small Scientists in similar positions.