How to make buildbot run the task by certain event?

Question

Is there any way to configure master.cfg as to schedule a builder to be executing the code by the time the project folder or file in it has been modified(or its FileVersionInfo changed)?

I know it is quite of a workaround, but i need this as to run the Build Verification Tests right after developers have compiled a newer version of a product, and since developing of a number of projects are not in CVS, I need a scanner for project folders.

Here are the extracts from master.cfg:

def create_builders_and_schedule_it(factory, build_slaves, **kwargs):
    # kwargs: change_filter=None, nightly=None, trigger_name=None)

    builder_names = []

    if hasattr(factory, 'subcategory'):
        cat = "%s_%s" % (factory.category, factory.subcategory)
    else:
        cat = factory.category

    if 'change_filter' in kwargs:
        change_filter = kwargs['change_filter']
    else:
        change_filter = filter.ChangeFilter( branch = factory.branch )

    for i_slave in build_slaves:
        builder_name = "%s_%s" % (cat, i_slave)
        builder_names.append(builder_name)
    c['builders'].append(
        BuilderConfig(name = builder_name,
                      category = factory.category,
                      factory = factory,
                      slavename = i_slave
                ))

    # every task has 'forced' scheduler at least
    c['schedulers'].append(
        ForceScheduler(name = cat + "_force",
                       builderNames = builder_names))

    # every task has 'on commit' scheduler
    c['schedulers'].append(
        SingleBranchScheduler(name = cat + "_singlebranch",
                       builderNames = builder_names,
                       change_filter = change_filter))

    # some task can be scheduled
    if 'nightly' in kwargs:
        c['schedulers'].append( timed.Nightly(
                                name = cat + '_nightly',
                                branch = factory.branch,
                                builderNames = builder_names,
                                hour = kwargs['nightly'][0], minute = kwargs['nightly'][1]))                                

    if 'trigger_name' in kwargs:
        c['schedulers'].append(
            Triggerable(
                name = kwargs['trigger_name'],
                builderNames = builder_names))

There's a typical class derived from BuildFactory:

class Autotests(BuildFactory):
    """ Build products from Autotests repository """
    def __init__(self, branch):
        BuildFactory.__init__(self)
        self.category = "autotests"
        self.branch = branch
        self.addStep(
            Git(repourl = AUTOTESTS_REPO, branch = branch))

    def build_source_archive(self):
        self.subcategory = "source"
        self.addStep(
            ShellCommand(
                workdir = "build",
                command = ["./upload_to_binary_repo.sh"]))
        return self

    def run_BVT_prod(self):
        self.subcategory = "BVT_prod"
        self.addStep(
            ShellCommand(
                workdir = "build/BVT_tests/prod",
                env = {'PATH':"/usr/bin:/bin", 'VIRTUAL_ENV':''},
                command = ["./bvt_runner.sh"] ))
        return self

Now as we have the method and the class, we

create_builders_and_schedule_it(
    Autotests("devel").run_BVT_Prod(),
    ['VIX_runner'],
    nightly = (2,0)
)

Is there a common way to perform this kind of check? Or should I look in another direction?

Answer 1

You can use a customized version of FileExists , something on the following lines:

from buildbot.steps.slave import FileExists
class CustomFileExists(FileExists):
    # By default false
    output_files_exist = False

    def __init__(self, file, **kwargs):
        CustomFileExists.output_files_exist = False
        FileExists.__init__(self, file, **kwargs)

    # Override the commandComplete
    def commandComplete(self, cmd):
        if not cmd.didFail():
            # for e.g. check the last modification time on the file
            # or do your own cooking to check whether to proceed or not !!
            s = cmd.updates["stat"][-1]
            if s[stat.ST_MTIME] > s[stat.ST_CTIME]:
                # If it satisfies your condition, then set it to True
                CustomFileExists.output_files_exist = True
        self.finished(SUCCESS)
        return

From the name it seems it just checks for the file but you can do whatever with the file, eg with the stat command.

In your main build configuration for this builder you need to add a new step before you execute the tests. Something like:

project_file = r'path\to\the\built\product\image'
factory.addStep(CustomFileExists(project_file))

If the file is not found (or didn't satisfy your constraints), it will stop there and tests will not be executed.

Important to note that this step will be executed on the slave, meaning the file will be checked on the slave and not on the master.

I hope this helps you.