[CIG-CS] updating CIG buildbots to use Git

[Brad Aagaard]

Matt,

I thought it would be simple too, but it isn't. Here is the current 
buildbot implementation for git:
https://github.com/buildbot/buildbot/blob/master/master/buildbot/steps/source/git.py

Here is the current CIG source code. Note the differences among SVN, 
Mercurial, and Git.

http://www.geodynamics.org/wsvn/cig/cs/buildbot/trunk/buildbot/steps/source.py?op=file&rev=0&sc=0

http://www.geodynamics.org/wsvn/cig/cs/buildbot/trunk/buildbot/repositories.py?op=file&rev=0&sc=0

Brad

On 04/18/2013 02:35 PM, Matthew Knepley wrote:
> On Thu, Apr 18, 2013 at 5:29 PM, Eric Heien <emheien at ucdavis.edu> wrote:
>
>> Hi Brad,
>>
>> The current CIG buildbot version is 0.7.5 dating from Dec. 2006 whereas
>> the current official buildbot version is 0.8.7.  The CIG system also uses
>> Python 2.4 and other outdated libraries.  From what I read on various
>> forums, the compatibility between 0.7.x slaves and a 0.8.x master is
>> questionable.  If we upgrade I'd like to completely wipe the old buildbot
>> structure and use a completely updated version.
>>
>> None of the other CIG codes are currently using the buildbot
>> infrastructure so whether we can do this depends on PyLith and how much
>> work it takes to customize buildbot.  Do you know how much customization
>> was involved in setting up the PyLith buildbots?  Do you remember what
>> needed to be customized?  I've been playing with the BaTLab system where
>> compiling external libraries is very simple, so I would imagine buildbot
>> should have similar ease of use and the customization was involved
>> elsewhere.
>>
>> I'd be willing to work with you, Charles and Matt over the next month to
>> get this working on an updated buildbot system since we'll need it anyway
>> as we move other codes to Git.  Let me know what you think,
>>
>
> Sure. I would think this would be easy since 'hg pull -u' and 'git checkout
> <branch>; git pull' will do the same things for a buildbot.
>
>     Matt
>
>
>> -Eric
>>
>> On Apr 18, 2013, at 1:55 PM, Brad Aagaard wrote:
>>
>>> QUESTION #1: Should CIG update to the latest buildbot release and
>>> recustomize it or try to update the current customized buildbot so that
>>> it has a more complete implementation for Git?
>>>
>>> QUESTION #2: Does CIG have the expertise and resources to complete this
>>> task in 4-6 weeks?
>>>
>>> Background
>>>
>>> I have messed with the buildbot configuration and source code to see if
>>> it is easy to update to support Git. It is difficult to tell how many
>>> obstacles are lurking under the hood.
>>>
>>> When the CIG buildbots migrated to UCDavis, we limited them to PyLith
>>> builds (although I think Walter used them to build Gale binaries at
>>> release time). They build the three main PyLith software components that
>>> change on a frequent basis- PyLith, petsc-dev, and spatialdata. We now
>>> need to update the buildbots to use Git for PETSc rather than mercurial.
>>> Additionally, we use the knepley/pylith branch of PETSc, so we need
>>> relatively up-to-date support for Git.
>>>
>>> The CIG buildbots run a customized version of buildbot that has not been
>>> kept in sync with buildbot releases. As a result, it appears to provide
>>> very limited support for Git repositories. This leads to the question of
>>> how to proceed with updating the buildbots to use Git.  Most of the
>>> customization was done when CIG started, so I am not sure what was done.
>>> I have done some customization, primarily to allow us to package a
>>> PyLith windows binary.
>>>
>>> I took over the buildbots to get them up and running when CIG moved to
>>> UCDavis, primarily because CIG had very limited staff and I know all the
>>> interdependencies for PyLith. However, I really don't have time to
>>> update buildbot. I would like CIG to do this but we can't wait 2 months
>>> for this to be done. We need to get a PyLith bugfix release out in 3-4
>>> weeks and a major new feature done by mid-June.
>>>
>>> Thanks,
>>> Brad
>>> _______________________________________________
>>> CIG-CS mailing list
>>> CIG-CS at geodynamics.org
>>> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-cs
>>
>>
>
>