[CIG-CS] updating CIG buildbots to use Git

[Brad Aagaard]

Eric,

I ran svn log in the buildbot top-level directory. Most of the 
customization appears to be for creating binaries. Some of the 
customization may no longer be needed if the newer buildbot release 
support things (like packaging binaries) that Leif added.

Moving to the updated release would definitely require starting from a 
clean slate. It may take a fair amount of work to figure out exactly 
what would need customization and how to do it given the significant 
differences between the current buildbot and the CIG buildbot source.

Brad


On 04/18/2013 02:29 PM, Eric Heien wrote:
> Hi Brad,
>
> The current CIG buildbot version is 0.7.5 dating from Dec. 2006 whereas the current official buildbot version is 0.8.7.  The CIG system also uses Python 2.4 and other outdated libraries.  From what I read on various forums, the compatibility between 0.7.x slaves and a 0.8.x master is questionable.  If we upgrade I'd like to completely wipe the old buildbot structure and use a completely updated version.
>
> None of the other CIG codes are currently using the buildbot infrastructure so whether we can do this depends on PyLith and how much work it takes to customize buildbot.  Do you know how much customization was involved in setting up the PyLith buildbots?  Do you remember what needed to be customized?  I've been playing with the BaTLab system where compiling external libraries is very simple, so I would imagine buildbot should have similar ease of use and the customization was involved elsewhere.
>
> I'd be willing to work with you, Charles and Matt over the next month to get this working on an updated buildbot system since we'll need it anyway as we move other codes to Git.  Let me know what you think,
>
> -Eric
>
> On Apr 18, 2013, at 1:55 PM, Brad Aagaard wrote:
>
>> QUESTION #1: Should CIG update to the latest buildbot release and
>> recustomize it or try to update the current customized buildbot so that
>> it has a more complete implementation for Git?
>>
>> QUESTION #2: Does CIG have the expertise and resources to complete this
>> task in 4-6 weeks?
>>
>> Background
>>
>> I have messed with the buildbot configuration and source code to see if
>> it is easy to update to support Git. It is difficult to tell how many
>> obstacles are lurking under the hood.
>>
>> When the CIG buildbots migrated to UCDavis, we limited them to PyLith
>> builds (although I think Walter used them to build Gale binaries at
>> release time). They build the three main PyLith software components that
>> change on a frequent basis- PyLith, petsc-dev, and spatialdata. We now
>> need to update the buildbots to use Git for PETSc rather than mercurial.
>> Additionally, we use the knepley/pylith branch of PETSc, so we need
>> relatively up-to-date support for Git.
>>
>> The CIG buildbots run a customized version of buildbot that has not been
>> kept in sync with buildbot releases. As a result, it appears to provide
>> very limited support for Git repositories. This leads to the question of
>> how to proceed with updating the buildbots to use Git.  Most of the
>> customization was done when CIG started, so I am not sure what was done.
>> I have done some customization, primarily to allow us to package a
>> PyLith windows binary.
>>
>> I took over the buildbots to get them up and running when CIG moved to
>> UCDavis, primarily because CIG had very limited staff and I know all the
>> interdependencies for PyLith. However, I really don't have time to
>> update buildbot. I would like CIG to do this but we can't wait 2 months
>> for this to be done. We need to get a PyLith bugfix release out in 3-4
>> weeks and a major new feature done by mid-June.
>>
>> Thanks,
>> Brad
>> _______________________________________________
>> CIG-CS mailing list
>> CIG-CS at geodynamics.org
>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-cs
>
> _______________________________________________
> CIG-CS mailing list
> CIG-CS at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-cs
>