Getting Started

Main concepts

Your project depends on other projects to build or run. In order to keep these dependencies together with your own code, Dfetch can fetch these and place them inside your project. For clarity lets call your project the superproject and the dependencies the subprojects.

Dfetch operates on a superproject: the project that contains the Manifest file. The superproject depends on one or more subprojects, which are the external projects listed in the manifest. The superproject itself can use any version control system (or even none at all).

The Manifest file describes all the Projects the superproject depends on. These subprojects can be a mix of different version control systems, such as Git or Subversion. You can then let Dfetch Update your dependencies based on this manifest through git or svn.

Dfetch will fetch the correct version of each subproject and place it in the location of your choice. If the destination folder already exists and the version has changed, Dfetch will overwrite the folder with the updated contents.

Since any version control information (such as .git or .svn directories) is removed from the fetched subprojects, Dfetch stores general information about each subproject in a .dfetch_data.yaml file inside the fetched directory, referred to as the metadata.

After updating, you can then review the changes using the version control system of your superproject and commit them as you see fit.

My first manifest

For a complete description of the manifest, see Manifest.

Dfetch can generate a basic manifest as starting point with dfetch init. Alternatively, Dfetch can generate a manifest based on the git submodules or svn externals in an existing project using dfetch import.

Below manifest will retrieve the given tag of mymodule from the url listed. For more options such as branch and revision see Manifest.

manifest:
  version: '0.0'

  projects:
    - name: ext/test-repo-tag
      tag: v1
      url: https://github.com/dfetch-org/test-repo

My first update

After creating the manifest we can let Dfetch perform an update. Make sure that you have installed Dfetch as described in Installation.

From a command-line perform the following command

$ dfetch update

Dfetch will search through all directories down until it finds the dfetch.yaml manifest and it will perform the update.

After Dfetch finishes, the version of the dependency as listed in your manifest is downloaded at the target location. You can now commit it to your version control system.

Inside the project folder, Dfetch will add a metadata file .dfetch_data.yaml containing information needed for knowing what version is present. Dfetch can function perfectly without this file, but since it will have no knowledge of the current contents, updates will always just blindly update.

My first version change

During development of your project you can periodically check for updates with:

$ dfetch check

Dfetch will check for each project if a newer version is available.

Dfetch (0.5.1)
  ext/test-repo-tag   : wanted & current (v1), available (v2.0)

As can be seen a newer tag is available. If you want to update, you can manually update the tag of the project in the manifest.

Note

If you only have a branch specified, Dfetch will update automatically.

manifest:
  version: '0.0'

  projects:
    - name: ext/test-repo-tag
      url: https://github.com/dfetch-org/test-repo
      tag: v2.0

And after that rerunning update:

$ dfetch update

Now you can review the changes and commit them once again if you are happy.

My first remote

Typically your project will have multiple dependencies. For instance take the below manifest.

manifest:
  version: '0.0'

  projects:
    - name: ext/test-repo-tag
      url: https://github.com/dfetch-org/test-repo

    - name: cpputest
      url: https://github.com/cpputest/cpputest

Both projects have a very similar url. To simplify your manifest, it is possible to create a single remote with the common part of the URL (url-base). Below manifest is completely equivalent to the manifest above. For more details on working with remotes see Remotes.

manifest:
  version: '0.0'

  remotes:
    - name: github
      url-base: https://github.com/

  projects:
    - name: ext/test-repo-tag
      repo-path: dfetch-org/test-repo

    - name: cpputest
      repo-path: cpputest/cpputest

Example: Basic user journey

../features/journey-basic-usage.feature

Given the manifest 'dfetch.yaml'
    """
    manifest:
      version: '0.0'

      projects:
        - name: ext/test-repo-tag
          tag: v1
          url: https://github.com/dfetch-org/test-repo
    """
When I run "dfetch update"
Then the following projects are fetched
    | path                    |
    | ext/test-repo-tag       |
When I run "dfetch check"
Then the output shows
    """
    Dfetch (0.13.0)
      ext/test-repo-tag:
      > wanted & current (v1), available (v2.0)
    """
When the manifest 'dfetch.yaml' is changed to
    """
    manifest:
      version: '0.0'

      projects:
        - name: ext/test-repo-tag
          url: https://github.com/dfetch-org/test-repo
          tag: v2.0
    """
And I run "dfetch update"
Then the following projects are fetched
    | path                    |
    | ext/test-repo-tag       |

My first patch

Sometimes you need to patch a dependency to get it working with your project. Dfetch supports creating and applying patches after fetching the dependency.

Given you have fetched dfetch-org/test-repo and you need to apply a patch. First commit the fetched version to your version control system. This provides Dfetch a reference point.

$ git add ext/test-repo-tag
$ git commit -m "Add test-repo-tag v2.0"

Then you can work on the files inside ext/test-repo-tag, once you are happy you can run dfetch diff to create a patch file with respect to the committed version.

$ dfetch diff

A patch file ext-test-repo-tag.patch is created in the current folder. You can place it anywhere you want, as long as it is reachable when running Dfetch. For this example we keep it in the current folder. This patch file can now be applied automatically by Dfetch in next updates. Add it to your manifest as shown below.

manifest:
  version: '0.0'

  remotes:
    - name: github
      url-base: https://github.com/

  projects:
    - name: ext/test-repo-tag
      repo-path: dfetch-org/test-repo
      patch: ext-test-repo-tag.patch

    - name: cpputest
      repo-path: cpputest/cpputest

To test the patch, update the project again but with the -f option to force overwriting the local changes. The local changes will be lost but the patch will be applied.

$ dfetch update -f ext/test-repo-tag

Now amend the last commit to include the fetched project and the patch file:

$ git add ext/test-repo-tag ext-test-repo-tag.patch
$ git commit --amend --no-edit

For more details on working with patches see Diff and Patch.

Example: Basic patch journey

../features/journey-basic-patching.feature

Given a local git repo "MyPatchExample" with the manifest
    """
    manifest:
      version: '0.0'

      projects:
        - name: test-repo
          dst: ext/test-repo
          tag: v1
          url: https://github.com/dfetch-org/test-repo
    """
When I run "dfetch update test-repo" in MyPatchExample
Then the following projects are fetched
    | path                           |
    | MyPatchExample/ext/test-repo   |
When all files in MyPatchExample/ext/test-repo are committed
And "ext/test-repo/my-new-file.md" in MyPatchExample is created
When I run "dfetch diff test-repo" in MyPatchExample
Then the patch file 'MyPatchExample/test-repo.patch' is generated
    """
    diff --git a/my-new-file.md b/my-new-file.md
    new file mode 100644
    index 0000000..0ee3895
    --- /dev/null
    +++ my-new-file.md
    @@ -0,0 +1,1 @@
    +Some content
    """
When the manifest 'dfetch.yaml' in MyPatchExample is changed to
    """
    manifest:
      version: '0.0'

      projects:
        - name: test-repo
          dst: ext/test-repo
          tag: v1
          url: https://github.com/dfetch-org/test-repo
          patch: test-repo.patch
    """
And I run "dfetch update -f test-repo"
Then the output shows
    """
    Dfetch (0.13.0)
      test-repo:
      > Fetched v1
      > Applying patch "test-repo.patch"
    """