How to build an efficient maven/jenkins build queue -

we use jenkins our ci system , maven our build tool. have svn repository our code , artifactory our artefacts.

the source code organized multi-module maven project. modules contribute actual application, modules applications tht need during build , test process.

we have multiple jobs for:

• a quick set of commit tests
• more extensive integration tests
• code coverage , static code analysis
• deployment test environment
• smoke tests test environment

one weakness of our build process compile different parts of project in different jobs, of parts on , on again. instead prefer build once deploy artifactory , use these artifacts else.

but don't know how make sure that:

• the size of artifactory doesn't explode, because dump hundreds of jars every day in it
• downstream builds use exact set of artifacts produced last upstream build job not weird mixture of version, because runs concurrently next build job, might have deployed new version of artifact not of artifact b.

any help, pointers or ideas appreciated.

there many ways answer question. using same set in development environment.

• you can use locks , latches plugin make sure things don't execute concurrently when need it. https://wiki.jenkins-ci.org/display/jenkins/locks+and+latches+plugin
• mix maven incremental , parallel builds speed individual builds.
• add jenkins slaves more 'executors'.
• to avoid artifactory bloat in snapshot repository - artifactory can configured keep number of snapshots.
• note: avoid using 'non-unique' snapshot repositories in artifactory , maven 3. see http://wiki.jfrog.org/confluence/display/rtf/local+repositories.
• for less changed modules, make release versions. keep in artifactory. use maven release plugin deploy artifactory , make proper version control tags (except if using git, doesn't play nice release plugin).
• use jenkins "disable automatic artifact archiving" snapshot artifacts want propagate via local repository in jenkins.
• use '-u' option on maven builds make sure latest snapshots used.

my personal preference keep simple: make single maven module whole application, infrequently changing code put separate module tree / job. simplicity, keep number of jobs minimum.

we tried using separate job testing, made tricky trace source of test failure, , made 'branching prod' more difficult.