Atomist’s ability to manipulate code and respond to events can help you and your team fix problems in your projects and ensure they stay fixed. Let’s explore this with a real example: renaming test files. You’ll see how to set up an Atomist open source software delivery machine (SDM) to do something useful on your local machine within minutes.

The Problem

At Atomist, we recently switched from a convention of naming tests ThingTest.ts to Thing.test.ts. We’ve updated some of our repos manually, but some of the bigger ones remain. One is the SDM repo itself. Some repos contain hundreds of test files, so this is a tedious job that should be automated.

Making code changes across many files and/or repositories is a job for an Atomist code transform.

Before we look at the code to do this, let’s make sure Atomist is set up so you can follow along.

Setting up the Atomist CLI

If you haven’t yet installed the Atomist CLI, please do so using npm to install it globally. This will give you the atomist command that you’ll use for local interactions with Atomist.

$ npm i -g @atomist/cli

(You’ll need to have Node and git installed.)

In a fresh terminal window, type atomist feed to see output from local SDM activity. A regular system terminal is fine, but if you’re on OS/X, you’ll get nicer rendering if you use iTerm2.

Next, create a new Atomist software delivery machine that will run the automations we’ll create. The CLI can help you to do this, creating a local git repo:

$ atomist create sdm

At the prompts, choose “blank” for SDM type and whatever name you like for the target repository. I chose “test-blog”. For “target-owner” choose whatever GitHub org you’d be likely to add this to. I chose “atomist-blogs”.

In the newly created directory — ~/atomist/<target-owner>/<target-repo> — type npm i to install the dependencies. Then check everything’s working by runningatomist start --local to start the SDM process. Open the project in your preferred IDE. I used VS Code, starting it by typingcode ..

We’ll also need one or more target repos to run our automation on. Atomist provides a convenient “clone” command that takes the same arguments as git clone and also installs the git hooks Atomist uses to raise events. Unlike git clone, this can be run in any directory, as it clones the project under Atomist’s code root — normally ~/atomist. I typed the following to clone the sdm project. Because this transform has since run for real and the project has moved on, check out the commit before the change was made:

$ atomist clone https://github.com/atomist/sdm $ cd ~/atomist/atomist/sdm $ git checkout 80231ad455cd072e7c8d06b46195bf10012583a6

Updating a Project With Atomist

We’ll need to add an Atomist code transform command to our new SDM to teach it to make the necessary changes. A code transform centers around a transform function running against the Atomist Project API. Atomist will take care of cloning and committing any changes made.

Here’s a stab at the required functionality, replacing all the filename suffixes of Test.ts with .test.ts. (Could you do this in Bash? Yes. But bear with me and you’ll see where Bash tends to fall short in solving such problems.)

If you’re impatient, here’s the finished project.

I put this code in lib/machine/machine.ts, which is where we add functionality to the SDM object. Find the whole file here; copy it into your SDM. To test it:

• If the SDM is running, stop it with Ctrl-C.
• Run atomist start --local in your SDM’s directory; this will compile the code and then run it.
• Over in ~/atomist/atomist/sdm (the root of the target project we want to change), run atomist update test filenames. That corresponds to the intent configured in the code transform.
• Look for a new branch with git branch . You should have a branch named like transform-standardize_test_filenames-1536190990341 (except a different generated number at the end).
• Check out the branch withgit checkout <branch name> , then try git showto see all the renames.
• Did it work? Try compiling the target project: npm run compile

The files were renamed, but we’re not quite there. There are compile errors like this:

Some of the test files we renamed were imported by other test files for their utility functions. We need to change some imports, too. Let’s try adding a regular expression for that. (Could you do this in Bash? Yes, technically. But keep going.)

Here is the same code transform with the additional step of replacing some import lines:

The code is here if you want to cut and paste it.

• Change the code and save
• In the sdm project, restart the SDM (Ctrl-C and then atomist start --local again)
• In the target project, get back to the starting point again: git checkout before-dot-test-dot-ts
• Run the editor again. atomist update test filenames
• Look for a new branch. Check that one out (and see the changes with git show).
• Compile with npm run compile.

It still doesn’t work. The compile errors are fewer and different:

It turns out that there are some files in thesrc tree whose names end in Test. We shouldn’t change those imports. I tried to get this working with a regular expression, but it started getting ugly. This isn’t a good fit for regular expressions.

Fortunately, Atomist has a more powerful idiom for working with code. Code is not just any text — it has a structure based on a grammar. That structure is called the AST (Abstract Syntax Tree). Atomist can integrate with multiple grammars, including TypeScript, Java, and Kotlin.

Using the AST, we can get at precisely the part of the code we care about: the string literal inside each import statement. We want to check each of these values; if it does not contain /src and does end with Test, we want to change it.

Using the TypeScript AST we can use an XPath-like path expression to select the string literal from each import under the test directory and decide whether or not to modify it. You can’t do this in Bash. Code transforms in Atomist can go far beyond the trivial.

I extracted the code transform into a function. It now looks like this:

The code is here; you’ll probably want to copy the file to get the imports.

When we repeat the steps to test this change, it compiles without error. The files are renamed and only the correct import statements are changed.

Keeping It Fixed

We’ve succesfully updated 37 files in this project, and can run the same code against others. And there’s something even better we can do. Suppose someone forgets the new test naming convention and adds a file in the old style. We can make sure our convention change sticks, as Atomist supports the notion of an autofix.

With autofixes, Atomist can ensure that errors never creep back in. This powerful way of expressing policy is possible because of Atomist’s innovative combination of event handling and code manipulation.

When we enable autofixes, Atomist can keep applying our transform as needed. To do this, we need to set a goal to tell Atomist to autofix on any pushes we care about:

sdm.addGoalContributions(onAnyPush().setGoals(AutofixGoal));

Autofixes work locally on any commit. But to show the power of policy across an organization, I’ll connect my SDM to the Atomist service in the cloud. I’ll start Atomist without the local flag, via atomist start. There’s no need to change a line of code; the API is the same.

To test this, let’s deliberately create a new test file with an old style name and push it to the blog2 branch. Atomist immediately makes an autofix and I see this in Slack:

An additional commit from Atomist renames the new file:

Our test naming problem is solved once and for all. This autofix will work locally, adding a commit on the same branch.

We can also execute transforms using Slack, against one or many repositories. This screenshot is the result of typing the same command I used in the CLI into the Atomist community Slack team, which is associated with the atomist GitHub organization. In the cloud, transform changes are presented in a pull request:

Summary

An Atomist code transform command has significant advantages over shell scripting:

• It’s discoverable. If you’d used a shell script, your colleagues wouldn’t know. Most likely, even if you wanted to do something similar again, it wouldn’t have been version controlled, so you’d have to start from scratch. This kind of one-off hacking isn’t how we develop business applications and it shouldn’t be how we develop our software, either.
• It can go beyond the limits of regular expressions. Atomist can use regular expressions, but its parsing support, combined with the power of TypeScript, mean the sky’s the limit.
• Atomist code transforms are testable. It’s easy to write unit tests to ensure that transforms are safe before letting them loose on your source code.
• Code transforms can run at scale. It’s easy to run transforms across many repositories, without needing to clone and push individually. You can run either locally or in the cloud against GitHub or Bitbucket.
• Code transforms are reusable. Because they’re written in a full-featured programming language using a powerful API, you can share your transforms with others, who can build on them.

But the biggest single advantage is the intersection with Atomist eventing, giving the ability to maintain consistency across an organization.

Autofixes are one example of how Atomist offers a strategic approach to delivery and automation, replacing one-off hacks with engineered policies.

You can use this approach to avoid many classes of error. At Atomist, we use autofixes to manage license headers and contractually required dependency listings. Our users do a wide range of things like avoiding security risks and idioms they wish to move away from, but which aren’t captured by traditional tools. You can also integrate third-party products like linters or static analysis tools.

What would you like to fix and keep fixed?