[CIG-CS] updating CIG buildbots to use Git

[Matthew Knepley]

On Thu, Apr 18, 2013 at 5:42 PM, Brad Aagaard <baagaard at usgs.gov> wrote:

> Matt,
>
> I thought it would be simple too, but it isn't. Here is the current
> buildbot implementation for git:
> https://github.com/buildbot/**buildbot/blob/master/master/**
> buildbot/steps/source/git.py<https://github.com/buildbot/buildbot/blob/master/master/buildbot/steps/source/git.py>
>
> Here is the current CIG source code. Note the differences among SVN,
> Mercurial, and Git.
>
> http://www.geodynamics.org/**wsvn/cig/cs/buildbot/trunk/**
> buildbot/steps/source.py?op=**file&rev=0&sc=0<http://www.geodynamics.org/wsvn/cig/cs/buildbot/trunk/buildbot/steps/source.py?op=file&rev=0&sc=0>
>
> http://www.geodynamics.org/**wsvn/cig/cs/buildbot/trunk/**
> buildbot/repositories.py?op=**file&rev=0&sc=0<http://www.geodynamics.org/wsvn/cig/cs/buildbot/trunk/buildbot/repositories.py?op=file&rev=0&sc=0>


Git and Mercurial look pretty close. So far, I am not worried. (Famous last
words)

   Matt


>
> Brad
>
>
> On 04/18/2013 02:35 PM, Matthew Knepley wrote:
>
>> On Thu, Apr 18, 2013 at 5:29 PM, Eric Heien <emheien at ucdavis.edu> wrote:
>>
>>  Hi Brad,
>>>
>>> The current CIG buildbot version is 0.7.5 dating from Dec. 2006 whereas
>>> the current official buildbot version is 0.8.7.  The CIG system also uses
>>> Python 2.4 and other outdated libraries.  From what I read on various
>>> forums, the compatibility between 0.7.x slaves and a 0.8.x master is
>>> questionable.  If we upgrade I'd like to completely wipe the old buildbot
>>> structure and use a completely updated version.
>>>
>>> None of the other CIG codes are currently using the buildbot
>>> infrastructure so whether we can do this depends on PyLith and how much
>>> work it takes to customize buildbot.  Do you know how much customization
>>> was involved in setting up the PyLith buildbots?  Do you remember what
>>> needed to be customized?  I've been playing with the BaTLab system where
>>> compiling external libraries is very simple, so I would imagine buildbot
>>> should have similar ease of use and the customization was involved
>>> elsewhere.
>>>
>>> I'd be willing to work with you, Charles and Matt over the next month to
>>> get this working on an updated buildbot system since we'll need it anyway
>>> as we move other codes to Git.  Let me know what you think,
>>>
>>>
>> Sure. I would think this would be easy since 'hg pull -u' and 'git
>> checkout
>> <branch>; git pull' will do the same things for a buildbot.
>>
>>     Matt
>>
>>
>>  -Eric
>>>
>>> On Apr 18, 2013, at 1:55 PM, Brad Aagaard wrote:
>>>
>>>  QUESTION #1: Should CIG update to the latest buildbot release and
>>>> recustomize it or try to update the current customized buildbot so that
>>>> it has a more complete implementation for Git?
>>>>
>>>> QUESTION #2: Does CIG have the expertise and resources to complete this
>>>> task in 4-6 weeks?
>>>>
>>>> Background
>>>>
>>>> I have messed with the buildbot configuration and source code to see if
>>>> it is easy to update to support Git. It is difficult to tell how many
>>>> obstacles are lurking under the hood.
>>>>
>>>> When the CIG buildbots migrated to UCDavis, we limited them to PyLith
>>>> builds (although I think Walter used them to build Gale binaries at
>>>> release time). They build the three main PyLith software components that
>>>> change on a frequent basis- PyLith, petsc-dev, and spatialdata. We now
>>>> need to update the buildbots to use Git for PETSc rather than mercurial.
>>>> Additionally, we use the knepley/pylith branch of PETSc, so we need
>>>> relatively up-to-date support for Git.
>>>>
>>>> The CIG buildbots run a customized version of buildbot that has not been
>>>> kept in sync with buildbot releases. As a result, it appears to provide
>>>> very limited support for Git repositories. This leads to the question of
>>>> how to proceed with updating the buildbots to use Git.  Most of the
>>>> customization was done when CIG started, so I am not sure what was done.
>>>> I have done some customization, primarily to allow us to package a
>>>> PyLith windows binary.
>>>>
>>>> I took over the buildbots to get them up and running when CIG moved to
>>>> UCDavis, primarily because CIG had very limited staff and I know all the
>>>> interdependencies for PyLith. However, I really don't have time to
>>>> update buildbot. I would like CIG to do this but we can't wait 2 months
>>>> for this to be done. We need to get a PyLith bugfix release out in 3-4
>>>> weeks and a major new feature done by mid-June.
>>>>
>>>> Thanks,
>>>> Brad
>>>> ______________________________**_________________
>>>> CIG-CS mailing list
>>>> CIG-CS at geodynamics.org
>>>> http://geodynamics.org/cgi-**bin/mailman/listinfo/cig-cs<http://geodynamics.org/cgi-bin/mailman/listinfo/cig-cs>
>>>>
>>>
>>>
>>>
>>
>>
>


-- 
What most experimenters take for granted before they begin their
experiments is infinitely more interesting than any results to which their
experiments lead.
-- Norbert Wiener
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://geodynamics.org/pipermail/cig-cs/attachments/20130418/0899de20/attachment.htm