A Step-by-Step Guide to Syncing a Private Repository to a Public one with Fastlane

Hello, there.

Publishing a Swift Package Manager (SPM) library involves a variety of steps to ensure that your code is accessible and usable by the developer community.

For projects that are developed in private repositories but need to be made public, syncing your private repository to a public GitHub repository is the crucial step.

This blog post will guide you through the process of releasing your SPM library to a public GitHub repository through a CI/CD pipeline, focusing on a Fastlane script designed to make the process as smooth as possible for the entire team of developers.

Introduction to CI/CD in Swift Package Manager development

CI/CD pipelines automate the steps of software release processes, from initial code commit to production deployment.

For SPM projects, this means ensuring that your library is always ready to be distributed and used by others, with minimal manual intervention.

Check the blog post about setting up a CI/CD pipeline for an SPM package, leveraging Fastlane, SwiftLint, SonarQube, and GitLab.

There I am addressing linting, testing and code analysis with SonarQube.

“A Mobile DevOps Story: Linking Private Workflows to Public Swift Packages”

Setting the stage for publishing

In this use-case, releasing your SPM library lies in the synchronization of your private development repository with your public GitHub repository.

This ensures that the latest version of your library is accessible to the community. Below, we break down each step of the release process and guide you through building a Fastlane lane to automate these tasks.

First define a new lane:

desc "Push the current branch to GitHub repository"
lane :push_to_github do
    # Define your GitHub repository and Access Token
    github_repo = ENV['GITHUB_REPOSITORY'] # format: username/repo
    github_token = ENV['GITHUB_TOKEN']

    # Verification of GitHub repo and token presence
    UI.error("GitHub repo not found in environment variables") unless github_repo
    UI.error("GitHub Token not found in environment variables") unless github_token

    version = "x.y.z" # Version should be dynamically retrieved from your system
    UI.error("Version not found.") unless version

    # ...
end

Preparing a new directory

This initial step involves creating a new directory outside your current private repository workspace. This directory will serve as a temporary workspace for the public repository you intend to update.

Of course, you could add a new remote to the already checkout repository, but if you want to:

• have a separate git history
• have internal sentisive files that shouldn’t be public
• have separate .gitignore files

then it’s easier to make sure that operations on the private repository do not interfere with your public repository's state, and vice versa.

desc "Push the current branch to GitHub repository"
lane :push_to_github do
    # ...

    github_clone_dir = "./../../github_clone_ios"
    sh "rm -rf #{github_clone_dir}"
    sh "mkdir #{github_clone_dir}"

    # ...
end

This code snippet clears any existing directory that may conflict with the new clone (rm -rf #{github_clone_dir}) and creates a fresh directory (mkdir #{github_clone_dir}) for cloning the public GitHub repository.

Cloning the Public Repository

Then, checkout your public repository into the newly created directory.

This step is crucial for ensuring that you are working with the most recent version of the public repository, thereby preventing any conflicts or overwrites.

desc "Push the current branch to GitHub repository"
lane :push_to_github do
    # ...

    # Clone the GitHub repo into the new directory
    git_url = "https://#{github_token}@github.com/#{github_repo}.git"
    Dir.chdir(github_clone_dir) do
        sh "git clone #{git_url} ."

        # Remove every file/directory that you want to be overriden.
        # By doing this, you avoid keeping deleted files after a copy
        sh "rm -rf Sources"
        sh "rm -rf Example" # this is the example app that integrates the library
    end

    # ...
end

Before you can sync the repositories, it's important to delete existing files from the checked-out public repository.

This cleanup step prevents outdated or unnecessary files from persisting after the sync, ensuring that the public repository mirrors the current state of your private repository.

For example, after a refactor you delete some files. When the project contents are copied to the public repo, the files that are deleted in the private one are still present. That is because there is no way in this approach to mark them to be deleted.

So the safest route is to delete the entire directories and just readd them.

Copying the project files

In this step, you copy your project files from the private repository to the public one.

This is the core of the syncing process, updating the public repository with the latest changes from your private repository.

Before executing this copy operation, ensure you've removed the .git folder and other version control-related files from the source directory to prevent overriding the public repository's Git history.

Of course, here it’s the time to remove sensitive and unnecessary files. For example, don’t copy the yml files or the Fastfile - related to the private CI/CD workflow.

desc "Push the current branch to GitHub repository"
lane :push_to_github do
    # ...

    # Remove the .git folder from the current directory
    # to not override the git history
    sh "rm -rf ../.git"
    sh "rm -rf ../.gitignore"

    # Remove sensitive and unncessarry files.
    # These are just some examples
    sh "rm -rf ../.build"
    sh "rm -rf ../fastlane"
    sh "rm -rf ../gitlab.yml"

    # Copy the contents to the cloned GitHub repo
    source = "../"
    destination = "#{github_clone_dir}"
    sh "cp -rf #{source} #{destination} "

    # ...
end

Committing changes

Once the files are copied, the next step is to commit these changes. This step prepares the files for pushing to the public repository, marking them as ready for the next release.

desc "Push the current branch to GitHub repository"
lane :push_to_github do
    # ...

    # Change directory to the cloned GitHub repo
    Dir.chdir(github_clone_dir) do
      # Add all changes
      sh "git add ."

      # Commit changes
      sh "git config user.email [email protected]"
      sh "git config user.name NameExample"
      sh "git commit -m '#{tag_name}'"
    end
end

This block adds all changes to Git's staging area and commits them with a message matching the version tag name.

Tagging the release

Tagging your release with a version number helps track different versions of your library and is essential for version control.

This tag will also be used to create a GitHub release, making it easier for users to find and use specific versions of your library.

desc "Push the current branch to GitHub repository"
lane :push_to_github do
    # ...

    # Change directory to the cloned GitHub repo
    Dir.chdir(github_clone_dir) do
        # ... committing changes

        # Create local tag
        # .strip! removes the whitespaces (spaces, tabs, newlines) in place
        version.strip!
        tag_name = "#{version}"
        sh "git tag #{tag_name}"

        # ...
    end
end

Pushing to Public repository

With the changes committed and tagged, push these changes to the public repository. This update makes the latest version of your library available to the public.

# Push changes to GitHub
sh "git push origin main --follow-tags"

This command pushes both the changes (commits) and the tags to the main branch of your public repository.

Creating a Github Release

This Fastlane action creates a GitHub release, providing a formal release entry that includes a downloadable source code zip file, tag, and release notes.

This step officially marks a new version of your library as released, providing users with detailed information about the update.

desc "Push the current branch to GitHub repository"
lane :push_to_github do
    # ...

    # Change directory to the cloned GitHub repo
    Dir.chdir(github_clone_dir) do
        # ... 
        # Create release on GitHub
        set_github_release(
          repository_name: github_repo,
          api_token: github_token,
          name: version,
          tag_name: tag_name,
          description: "#{tag_name} release",
        )
    end
end

Tips and tricks

When syncing your private repository with a public one, especially in the context of releasing a SPM library, there are nuances and potential pitfalls to be aware of.

Understand Git's Behavior

• Git History Preservation: Be mindful of the history in your public repository. When copying files from the private repository, the commit history won’t automatically transfer. This might be preferable for keeping your development history private but consider if there are critical version histories or contributions that need to be available publicly as well.
• Force Pushes: Avoid force pushing unless absolutely necessary, as it can overwrite history in the public repository, potentially causing loss of data for anyone who has forked or cloned the repository.

Managing Environmental Variables

• Security of Tokens: Your GitHub token is a sensitive piece of information. Always keep it secure by using environment variables or encrypted secrets, especially in CI/CD pipelines.
• Consistency Across Environments: Ensure that your environment variables are correctly set up and consistent across all development and CI/CD environments. A missing or misspelled environment variable can halt your release process.

Directory Management

• Relative Paths: Given that the working directory in Fastlane is the Fastlane directory itself, always use relative paths with this in mind. Misunderstanding this can lead to scripts not finding the correct files or directories, causing errors.
• Clean Directory Strategy: When preparing the public repository directory, a thorough cleanup is crucial. However, be specific about what you delete to avoid removing necessary files or directories unintentionally. Use explicit paths and double-check your commands.

Tagging and Releases

• Semantic Versioning: Stick to a consistent versioning scheme like semantic versioning. This makes it easier for users of your library to understand the changes between versions.
• Tag Descriptions: Use the tag description or release notes to communicate what changes are included in the release. This is especially useful for users to quickly grasp the impact of the update.

Test. A lot

• Local Lane Execution: Start by executing your Fastlane lane (push_to_github in this context) locally on your machine. This allows you to catch any immediate errors or misconfigurations in your script. Make sure you have a test public repository to push to, which can act as a sandbox for these tests to avoid cluttering your actual public repository with test data.
• Debugging: Utilize Fastlane's built-in logging and add additional print statements if necessary to understand the flow and output at each stage of your lane. Debugging information can be invaluable, especially when dealing with complex automation scripts.
• Keep your public repository private until you’re sure: Keeping your public repository private during the initial setup and testing phase of your CI/CD pipeline serves as a safe testing ground. You can run multiple syncs, test different configurations, and even make mistakes without the worry of exposing these trials to your end-users or having an impact on the perceived reliability of your library.
• Cleaning the Git History: Before making your repository public, it's an excellent time to clean up the git history. This could involve squashing unnecessary commits, removing sensitive data accidentally pushed, or reorganizing commits for clarity. Cleaning your git history helps in maintaining a professional and tidy project history that is easier for others to follow and understand.

Conclusion

Syncing a private SPM library project to a public GitHub repository doesn't have to be a daunting task.

I used this approach multiple times to ensures that my library/SDK is consistently up-to-date and available for public consumption by minimizing manual errors, making the release cycle efficient and smooth.