Manifest

A manifest file is a .yaml file describing what external projects are used in this project.

This can be any external repository (git, svn). A section remotes (see Remotes) which contains a list of sources of the projects to download and a section projects: that contains a list of projects to fetch.

manifest:
    version: 0.0

    remotes:
    - name: git-modules
      url-base: http://git.mycompany.local/mycompany/

    projects:
     - name: mymodule
       dst: external/mymodule/

Remotes

Remotes are the external repository where the code should be retrieved from.

The remotes: section is not mandatory. If only one remote is added this is assumed to be the default. If multiple remotes are listed default: can be explicitly specified. If multiple remotes are marked as default, the first marked as default is chosen.

manifest:
    version: 0.0

    remotes:
    - name: mycompany-git-modules
      url-base: http://git.mycompany.local/mycompany/
      default: true
    - name: github
      url-base: https://github.com/

Projects

Projects are a specific repository or sources to download from a remote location.

In its most basic form a project only has a name:. This would make Dfetch retrieve the mymodule project from the only remote listed (mycompany-git-modules) and place it in a folder mymodule in the same folder as the manifest.

A project name must be unique and each manifest must have at least one project.

manifest:
    version: 0.0

    remotes:
    - name: mycompany-git-modules
      url-base: http://git.mycompany.local/mycompany/

    projects:
     - name: mymodule

Revision/Branch/Tag

When no version is provided the latest version of the default branch (e.g. trunk, master, main) of a project will be chosen. Since we want more control on what project is retrieved the revision:, branch: and tag: attributes can help. Below manifest will download tag v1.13 of mymodule.

The tag: attribute takes priority over revision: and branch:. With revision: a specific commit (git) or revision (svn) is retrieved. For git, revisions must be complete 40 character long SHA-hashes. For git if only a revision is provided and no branch, DFetch cannot determine if there are updates, so it will assume you require that specific version.

manifest:
    version: 0.0

    remotes:
    - name: mycompany-git-modules
      url-base: http://git.mycompany.local/mycompany/

    projects:
     - name: mymodule
       tag: v1.13

     - name: myothermodule
       revision: dcc92d0ab6c4ce022162a23566d44f673251eee4

Note

For svn a standard layout is advised. Meaning the top of the repository has a trunk, branches and tags folder. If this is not the case, you can indicate this by using branch: ' '.

Destination

To control where the project is placed, the dst: attribute can be used. Below manifest will place mymodule at the relative path listed by dst: (relative to the manifest location).

manifest:
    version: 0.0

    remotes:
    - name: mycompany-git-modules
      url-base: http://git.mycompany.local/mycompany/

    projects:
     - name: mymodule
       dst: external/mymodule

If no dst: is provided, DFetch will use the project name as relative path. The dst: must be in the same folder or a folder below the manifest and must be unique.

Example: No fetch is done directly into manifest folder
../features/fetch-checks-destination.feature
Given the manifest 'dfetch.yaml'
    """
    manifest:
      version: '0.0'

      projects:
        - name: ext/test-repo-tag
          url: https://github.com/dfetch-org/test-repo
          tag: v1
          dst: .

    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      ext/test-repo-tag:
      > Skipping, path "." is not allowed as destination.
    Destination must be in a valid subfolder. "." is not valid!
    """
Example: Path traversal is not allowed
../features/fetch-checks-destination.feature
Given the manifest 'dfetch.yaml'
    """
    manifest:
      version: '0.0'

      projects:
        - name: ext/test-repo-tag
          url: https://github.com/dfetch-org/test-repo
          tag: v1
          dst: ../../some-higher-folder

    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      ext/test-repo-tag:
      > Skipping, path "../../some-higher-folder" is outside manifest directory tree.
    Destination must be in the manifests folder or a subfolder. "../../some-higher-folder" is outside this tree!
    """

Repo-path

When working with remotes by default Dfetch will take the url-base of the remote and concatenate that with the name of the project. Sometimes you want more control of the name, you can use the repo-path: attribute to list it explicitly.

For instance, below example will look for the remote project at <url-base>:<repo-path>, which would be https://github.com/cpputest/cpputest.

manifest:
    version: 0.0

    remotes:
    - name: github
      url-base: https://github.com/

    projects:
    - name: cpputest
      repo-path: cpputest/cpputest

Source

In larger projects or mono-repo’s it is often desirable to retrieve only a subfolder of an external project. Dfetch makes this possible through the src: attribute.

For instance if you are only interested in the src folder of cpputest you can limit the checkout to only that folder. Dfetch will retain any license file in the root of the repository.

Example: License is preserved in git repo sparse checkout and cannot be ignored
../features/keep-license-in-project.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeProjectWithLicense
              url: some-remote-server/SomeProjectWithLicense.git
              src: SomeFolder/
              tag: v1
              ignore:
                  - LICENSE
    """
And a git-repository "SomeProjectWithLicense.git" with the files
    | path                                  |
    | LICENSE                               |
    | SomeFolder/SomeFile.txt               |
    | SomeOtherFolder/SomeOtherFile.txt     |
    | SomeSubFolder/LICENSE                 |
When I run "dfetch update"
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProjectWithLicense/
            .dfetch_data.yaml
            LICENSE
            SomeFile.txt
        dfetch.yaml
    """
Example: License is preserved in svn repo sparse checkout and cannot be ignored
../features/keep-license-in-project.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeProjectWithLicense
              url: some-remote-server/SomeProjectWithLicense
              src: SomeFolder/
              ignore:
                  - LICENSE
    """
And a svn-server "SomeProjectWithLicense" with the files
    | path                                  |
    | LICENSE                               |
    | SomeFolder/SomeFile.txt               |
    | SomeOtherFolder/SomeOtherFile.txt     |
When I run "dfetch update"
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProjectWithLicense/
            .dfetch_data.yaml
            LICENSE
            SomeFile.txt
        dfetch.yaml
    """
Example: A single file is fetched from svn repo
../features/keep-license-in-project.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeProjectWithLicense
              dst: SomeFile.txt
              url: some-remote-server/SomeProjectWithLicense
              src: SomeFolder/SomeFile.txt
    """
And a svn-server "SomeProjectWithLicense" with the files
    | path                                  |
    | COPYING.txt                           |
    | SomeFolder/SomeFile.txt               |
    | SomeOtherFolder/SomeOtherFile.txt     |
When I run "dfetch update"
Then 'MyProject' looks like:
    """
    MyProject/
        .dfetch_data-SomeFile.txt.yaml
        COPYING.txt
        SomeFile.txt
        dfetch.yaml
    """

manifest:
    version: 0.0

    remotes:
    - name: github
      url-base: https://github.com/

    projects:
    - name: cpputest
      src: src
      repo-path: cpputest/cpputest

It is also possible to use an * to match only certain files with the src tag. The following manifest will only checkout files in folder src with the *.h extension.

manifest:
    version: 0.0

    remotes:
    - name: github
      url-base: https://github.com/

    projects:
    - name: cpputest
      src: src/*.h
      repo-path: cpputest/cpputest
Example: A file pattern is fetched from a repo
../features/fetch-file-pattern-git.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeProjectWithAnInterestingFile
              url: some-remote-server/SomeProjectWithAnInterestingFile.git
              src: SomeFolder/SomeSubFolder/*.txt
              tag: v1
    """
And a git-repository "SomeProjectWithAnInterestingFile.git" with the files
    | path                                   |
    | SomeFolder/SomeSubFolder/SomeFile.txt  |
    | SomeFolder/SomeSubFolder/OtherFile.txt |
    | SomeFolder/SomeSubFolder/SomeFile.md   |
    | SomeFolder/Unrelated.txt               |
    | AlsoUnrelated.txt                      |
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      SomeProjectWithAnInterestingFile:
      > Fetched v1
    """
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProjectWithAnInterestingFile/
            .dfetch_data.yaml
            OtherFile.txt
            SomeFile.txt
        dfetch.yaml
    """
Example: A file pattern matches two files in different subfolders
../features/fetch-file-pattern-git.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeProjectWithAnInterestingFile
              url: some-remote-server/SomeProjectWithAnInterestingFile.git
              src: SomeFolder/Some*
              tag: v1
    """
And a git-repository "SomeProjectWithAnInterestingFile.git" with the files
    | path                                       |
    | SomeFolder/SomeSubFolder/SomeFile.txt      |
    | SomeFolder/SomeOtherSubFolder/SomeFile.txt |
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
    The 'src:' filter 'SomeFolder/Some*' matches multiple directories from 'some-remote-server/SomeProjectWithAnInterestingFile.git'. Only considering files in 'SomeFolder/SomeOtherSubFolder'.
      SomeProjectWithAnInterestingFile:
      > Fetched v1
    """
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProjectWithAnInterestingFile/
            .dfetch_data.yaml
            SomeFile.txt
        dfetch.yaml
    """

Ignore

In larger projects or mono-repo’s it is often desirable to ignore certain files such as Tests Dfetch makes this possible through the ignore: attribute.

For instance if you are not interested in the tests folder of cpputest you can limit the checkout to ignore that folder. Dfetch will retain any license file in the root of the repository.

manifest:
    version: 0.0

    remotes:
    - name: github
      url-base: https://github.com/

    projects:
    - name: cpputest
      ignore:
      - tests
      repo-path: cpputest/cpputest

Note

For projects using src: field, the ignore list will be relative to that folder. And the ignores will be applied after the src: pattern was applied. License files will never be excluded, since you likely shouldn’t be doing that.

Example: A file pattern is fetched from a repo
../features/fetch-with-ignore-git.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeInterestingProject
              url: some-remote-server/SomeInterestingProject.git
              src: SomeFolder/SomeSubFolder
              ignore:
                 - OtherFile.txt
              tag: v1
    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      SomeInterestingProject:
      > Fetched v1
    """
Then 'MyProject' looks like:
    """
    MyProject/
        SomeInterestingProject/
            .dfetch_data.yaml
            SomeFile.md
            SomeFile.txt
        dfetch.yaml
    """
Example: Combination of directories and a single file can be ignored
../features/fetch-with-ignore-git.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeInterestingProject
              url: some-remote-server/SomeInterestingProject.git
              ignore:
                 - SomeFolder/SomeOtherSubFolder
                 - SomeFolder/SomeSubFolder/SomeFile.md
              tag: v1
    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      SomeInterestingProject:
      > Fetched v1
    """
Then 'MyProject' looks like:
    """
    MyProject/
        SomeInterestingProject/
            .dfetch_data.yaml
            SomeFolder/
                SomeSubFolder/
                    OtherFile.txt
                    SomeFile.txt
        dfetch.yaml
    """
Example: Ignore overrides the file pattern match in src attribute
../features/fetch-with-ignore-git.feature
Given the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
        version: 0.0
        projects:
            - name: SomeInterestingProject
              url: some-remote-server/SomeInterestingProject.git
              src: SomeFolder/SomeSubFolder/*.txt
              ignore:
                 - /SomeNonExistingPath
                 - SomeFile.*
              tag: v1
    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      SomeInterestingProject:
      > Fetched v1
    """
Then 'MyProject' looks like:
    """
    MyProject/
        SomeInterestingProject/
            .dfetch_data.yaml
            OtherFile.txt
        dfetch.yaml
    """

VCS type

DFetch does its best to find out what type of version control system (vcs) the remote url is, for instance by trying a simple call to the remote repository. But sometimes both are possible, for example, in the past GitHub provided an svn and git interface at the same url.

To provide you an option to explicitly state the vcs, the vcs: attribute was introduced. In the below example the same project is fetched as SVN and as Git repository. Dfetch will default to the latest revision from trunk for svn and master from git.

manifest:
    version: 0.0

    remotes:
    - name: github
      url-base: https://github.com/

    projects:
    - name: cpputest
      vcs: git
      repo-path: cpputest/cpputest

    - name: cpputestSVN
      vcs: svn
      repo-path: cpputest/cpputest

Archive

Projects distributed as .tar.gz, .tgz, .tar.bz2, .tar.xz or .zip archive files can be fetched using vcs: archive. DFetch downloads the archive from the url: and extracts it to the destination directory, stripping the top-level directory if present.

The src: and ignore: attributes work the same way as for git/SVN projects.

manifest:
    version: 0.0

    projects:
    - name: my-library
      vcs: archive
      url: https://example.com/releases/my-library-1.0.tar.gz

Integrity verification

Use the integrity: block to verify the integrity of the downloaded archive. Currently the hash: sub-field is supported (format <algorithm>:<hex-digest>); only sha256 is recognised today, but the block is designed to grow to support detached signature verification via sig: (signature URL) and sig-key: (signing-key URL or fingerprint) in the future.

manifest:
    version: 0.0

    projects:
    - name: my-library
      vcs: archive
      url: https://example.com/releases/my-library-1.0.tar.gz
      integrity:
        hash: sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Run dfetch freeze after an initial dfetch update to add the sha256 hash to the manifest automatically.

Example: Tar.gz archive project is fetched
../features/fetch-archive.feature
Given an archive "SomeProject.tar.gz" with the files
    | path       |
    | README.md  |
    | src/main.c |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
    """
When I run "dfetch update" in MyProject
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProject/
            .dfetch_data.yaml
            README.md
            src/
                main.c
        dfetch.yaml
    """
Example: Zip archive project is fetched
../features/fetch-archive.feature
Given an archive "SomeProject.zip" with the files
    | path          |
    | README.md     |
    | include/lib.h |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject
          url: some-remote-server/SomeProject.zip
          vcs: archive
    """
When I run "dfetch update" in MyProject
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProject/
            .dfetch_data.yaml
            README.md
            include/
                lib.h
        dfetch.yaml
    """
Example: Archive projects with sha256, sha384 and sha512 hash verification are fetched
../features/fetch-archive.feature
Given an archive "SomeProject.tar.gz" with the files
    | path      |
    | README.md |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject-sha256
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
          integrity:
            hash: sha256:<archive-sha256>
        - name: SomeProject-sha384
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
          integrity:
            hash: sha384:<archive-sha384>
        - name: SomeProject-sha512
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
          integrity:
            hash: sha512:<archive-sha512>
    """
When I run "dfetch update" in MyProject
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProject-sha256/
            .dfetch_data.yaml
            README.md
        SomeProject-sha384/
            .dfetch_data.yaml
            README.md
        SomeProject-sha512/
            .dfetch_data.yaml
            README.md
        dfetch.yaml
    """
Example: Archive with incorrect sha256 hash is rejected
../features/fetch-archive.feature
Given an archive "SomeProject.tar.gz" with the files
    | path      |
    | README.md |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
          integrity:
            hash: sha256:0000000000000000000000000000000000000000000000000000000000000000
    """
When I run "dfetch update" in MyProject
Then the output shows
    """
    Dfetch (0.13.0)
    Hash mismatch for SomeProject! sha256 expected 0000000000000000000000000000000000000000000000000000000000000000
    """
Example: Specific directory from archive can be fetched
../features/fetch-archive.feature
Given an archive "SomeProject.tar.gz" with the files
    | path              |
    | src/main.c        |
    | src/lib.c         |
    | tests/test_main.c |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
          src: src/
    """
When I run "dfetch update" in MyProject
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProject/
            .dfetch_data.yaml
            lib.c
            main.c
        dfetch.yaml
    """
Example: Files can be ignored when fetching from archive
../features/fetch-archive.feature
Given an archive "SomeProject.tar.gz" with the files
    | path              |
    | README.md         |
    | src/main.c        |
    | tests/test_main.c |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
          ignore:
            - tests
    """
When I run "dfetch update" in MyProject
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProject/
            .dfetch_data.yaml
            README.md
            src/
                main.c
        dfetch.yaml
    """
Example: Archive is re-fetched when force flag is given
../features/fetch-archive.feature
Given an archive "SomeProject.tar.gz" with the files
    | path      |
    | README.md |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
    """
And all projects are updated in MyProject
When I run "dfetch update --force" in MyProject
Then the output shows
    """
    Dfetch (0.13.0)
      SomeProject:
      > Fetched some-remote-server/SomeProject.tar.gz
    """
Example: Multiple archive projects are fetched
../features/fetch-archive.feature
Given an archive "LibA.tar.gz" with the files
    | path      |
    | README.md |
And an archive "LibB.zip" with the files
    | path      |
    | README.md |
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: LibA
          url: some-remote-server/LibA.tar.gz
          vcs: archive

        - name: LibB
          url: some-remote-server/LibB.zip
          vcs: archive
    """
When I run "dfetch update" in MyProject
Then the following projects are fetched
    | path           |
    | MyProject/LibA |
    | MyProject/LibB |
Example: Archive with internal relative symlink using .. is fetched safely
../features/fetch-archive.feature
Given an archive "SomeProject.tar.gz" with the files
    | path             |
    | README.md        |
    | other/target.mk  |
And the archive "SomeProject.tar.gz" contains a symlink "sub/dir/link.mk" pointing to "../../other/target.mk"
And the manifest 'dfetch.yaml' in MyProject
    """
    manifest:
      version: '0.0'
      projects:
        - name: SomeProject
          url: some-remote-server/SomeProject.tar.gz
          vcs: archive
    """
When I run "dfetch update" in MyProject
Then 'MyProject' looks like:
    """
    MyProject/
        SomeProject/
            .dfetch_data.yaml
            README.md
            other/
                target.mk
            sub/
                dir/
                    link.mk
        dfetch.yaml
    """
And 'MyProject/SomeProject/sub/dir/link.mk' is a symlink pointing to '../../other/target.mk'

Patch

DFetch promotes upstreaming changes, but also allows local changes. These changes can be managed with local patch files. DFetch will apply the patch files in order every time a new upstream version is fetched. The patch file can be specified with the patch: attribute. This can be a single patch file or multiple. Patch files should be UTF-8 encoded files and using forward slashes is encouraged for cross-platform support. The path should be relative to the directory of the manifest.

manifest:
    version: 0.0

    remotes:
    - name: github
      url-base: https://github.com/

    projects:
    - name: cpputest
      vcs: git
      repo-path: cpputest/cpputest
      patch: patches/local_changes.patch

The patch should be generated using the Dfetch Diff command. Alternately the patch can be generated manually as such and should be a relative patch, relative to the fetched projects root.

$ git diff --relative=path/to/project HEAD > my_patch.patch

For more details see the git-diff documentation.

Example: A patch file is generated
../features/diff-in-git.feature
Given "SomeProject/README.md" in MyProject is changed and committed with
    """
    An important sentence for the README!
    """
When I run "dfetch diff SomeProject"
Then the patch file 'MyProject/SomeProject.patch' is generated
    """
    diff --git a/README.md b/README.md
    index 1e65bd6..faa3b21 100644
    --- a/README.md
    +++ b/README.md
    @@ -1 +1,2 @@
    Generated file for SomeProject.git
    +An important sentence for the README!
    """
Example: New files are part of the patch
../features/diff-in-git.feature
Given files as '*.tmp' are ignored in git in MyProject
And "SomeProject/NEWFILE.md" in MyProject is created and committed with
    """
    A completely new tracked file.
    """
And "SomeProject/NEW_UNCOMMITTED_FILE.md" in MyProject is created
And "SomeProject/IGNORE_ME.tmp" in MyProject is created
When I run "dfetch diff SomeProject"
Then the patch file 'MyProject/SomeProject.patch' is generated
    """
    diff --git a/NEWFILE.md b/NEWFILE.md
    new file mode 100644
    index 0000000..a2d8605
    --- /dev/null
    +++ b/NEWFILE.md
    @@ -0,0 +1 @@
    +A completely new tracked file.

    diff --git a/NEW_UNCOMMITTED_FILE.md b/NEW_UNCOMMITTED_FILE.md
    new file mode 100644
    index 0000000..0ee3895
    --- /dev/null
    +++ NEW_UNCOMMITTED_FILE.md
    @@ -0,0 +1,1 @@
    +Some content

    """
Example: No change is present
../features/diff-in-git.feature
When I run "dfetch diff SomeProject"
Then the output shows
"""
Dfetch (0.13.0)
  SomeProject:
  > No diffs found since 59efb91396fd369eb113b43382783294dc8ed6d2
"""
Example: Diff is generated on uncommitted changes
../features/diff-in-git.feature
Given "SomeProject/README.md" in MyProject is changed with
    """
    An important sentence for the README!
    """
When I run "dfetch diff SomeProject"
Then the patch file 'MyProject/SomeProject.patch' is generated
    """
    diff --git a/README.md b/README.md
    index 1e65bd6..faa3b21 100644
    --- a/README.md
    +++ b/README.md
    @@ -1 +1,2 @@
    Generated file for SomeProject.git
    +An important sentence for the README!
    """
Example: Metadata is not part of diff
../features/diff-in-git.feature
Given the metadata file ".dfetch_data.yaml" of "MyProject/SomeProject" is changed
When I run "dfetch diff SomeProject"
Then the output shows
"""
Dfetch (0.13.0)
  SomeProject:
  > No diffs found since 59efb91396fd369eb113b43382783294dc8ed6d2
"""
Example: A patch file is applied after fetching
../features/patch-after-fetch-git.feature
Given the manifest 'dfetch.yaml'
    """
    manifest:
        version: '0.0'

        remotes:
        - name: github-com-dfetch-org
          url-base: https://github.com/dfetch-org/test-repo

        projects:
        - name: ext/test-repo-tag
          tag: v2.0
          dst: ext/test-repo-tag
          patch: diff.patch
    """
And the patch file 'diff.patch'
    """
    diff --git a/README.md b/README.md
    index 32d9fad..62248b7 100644
    --- a/README.md
    +++ b/README.md
    @@ -1,2 +1,2 @@
     # Test-repo
    -A test repo for testing dfetch.
    +A test repo for testing patch.
    """
When I run "dfetch update"
Then the patched 'ext/test-repo-tag/README.md' is
    """
    # Test-repo
    A test repo for testing patch.
    """
Example: Applying patch file fails
../features/patch-after-fetch-git.feature
Given the manifest 'dfetch.yaml'
    """
    manifest:
        version: '0.0'

        remotes:
        - name: github-com-dfetch-org
          url-base: https://github.com/dfetch-org/test-repo

        projects:
        - name: ext/test-repo-tag
          tag: v2.0
          dst: ext/test-repo-tag
          patch: diff.patch
    """
And the patch file 'diff.patch'
    """
    diff --git a/README.md b/README1.md
    index 32d9fad..62248b7 100644
    --- a/README1.md
    +++ b/README1.md
    @@ -1,2 +1,2 @@
     # Test-repo
    -A test repo for testing dfetch.
    +A test repo for testing patch.
    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      ext/test-repo-tag:
      > Fetched v2.0
      > Applying patch "diff.patch"
        source/target file does not exist:
          --- b'README1.md'
          +++ b'README1.md'
    Applying patch "diff.patch" failed
    """
Example: Multiple patch files are applied after fetching
../features/patch-after-fetch-git.feature
Given the manifest 'dfetch.yaml'
    """
    manifest:
        version: '0.0'

        remotes:
        - name: github-com-dfetch-org
          url-base: https://github.com/dfetch-org/test-repo

        projects:
        - name: ext/test-repo-tag
          tag: v2.0
          dst: ext/test-repo-tag
          patch:
           - 001-diff.patch
           - 002-diff.patch
    """
And the patch file '001-diff.patch'
    """
    diff --git a/README.md b/README.md
    index 32d9fad..62248b7 100644
    --- a/README.md
    +++ b/README.md
    @@ -1,2 +1,2 @@
     # Test-repo
    -A test repo for testing dfetch.
    +A test repo for testing patch.
    """
And the patch file '002-diff.patch'
    """
    diff --git a/README.md b/README.md
    index 62248b7..32d9fad 100644
    --- a/README.md
    +++ b/README.md
    @@ -1,2 +1,2 @@
     # Test-repo
    -A test repo for testing patch.
    +A test repo for testing dfetch.
    """
When I run "dfetch update"
Then the patched 'ext/test-repo-tag/README.md' is
    """
    # Test-repo
    A test repo for testing dfetch.
    """
And the output shows
    """
    Dfetch (0.13.0)
      ext/test-repo-tag:
      > Fetched v2.0
      > Applying patch "001-diff.patch"
        successfully patched 1/1:    b'README.md'
      > Applying patch "002-diff.patch"
        successfully patched 1/1:    b'README.md'
    """
Example: Fallback to other file encodings if patch file is not UTF-8 encoded
../features/patch-after-fetch-git.feature
Given the manifest 'dfetch.yaml'
    """
    manifest:
        version: '0.0'

        remotes:
        - name: github-com-dfetch-org
          url-base: https://github.com/dfetch-org/test-repo

        projects:
        - name: ext/test-repo-tag
          tag: v2.0
          dst: ext/test-repo-tag
          patch: diff.patch
    """
And the patch file 'diff.patch' with 'UTF-16' encoding
    """
    diff --git a/README.md b/README.md
    index 32d9fad..62248b7 100644
    --- a/README.md
    +++ b/README.md
    @@ -1,2 +1,2 @@
     # Test-repo
    -A test repo for testing dfetch.
    +A test repo for testing patch.
    """
When I run "dfetch update"
Then the patched 'ext/test-repo-tag/README.md' is
    """
    # Test-repo
    A test repo for testing patch.
    """
And the output shows
    """
    Dfetch (0.13.0)
      ext/test-repo-tag:
      > Fetched v2.0
      > Applying patch "diff.patch"
        error: no patch data found!
        successfully patched 1/1:    b'README.md'
      > After retrying found that patch-file "diff.patch" is not UTF-8 encoded, consider saving it with UTF-8 encoding.
    """
Example: Patch files are outside manifest dir
../features/patch-after-fetch-git.feature
Given the manifest 'dfetch.yaml'
    """
    manifest:
        version: '0.0'

        remotes:
        - name: github-com-dfetch-org
          url-base: https://github.com/dfetch-org/test-repo

        projects:
        - name: ext/test-repo-tag
          tag: v2.0
          dst: ext/test-repo-tag
          patch: ../diff.patch
    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      ext/test-repo-tag:
      > Fetched v2.0
      > Skipping patch "../diff.patch" which is outside /some/path.
    """
Example: Patch applies when upstream adds extra lines
../features/patch-fuzzy-matching-git.feature
Given "README.md" in git-repository "SomeProject.git" is changed to:
    """
    Generated file for SomeProject.git
    Additional upstream line.
    """
When I run "dfetch update"
Then the patched 'MyProject/SomeProject/README.md' is
    """
    Generated file for SomeProject.git
    An important sentence for the README!
    Additional upstream line.
    """
Example: Patch applies when context lines have changed
../features/patch-fuzzy-matching-git.feature
Given "README.md" in git-repository "SomeProject.git" is changed to:
    """
    Generated file for SomeProject
    This repository is used for testing.
    """
And the patch file 'SomeProject.patch' in MyProject
    """
    diff --git a/README.md b/README.md
    index 1e65bd6..faa3b21 100644
    --- a/README.md
    +++ b/README.md
    @@ -1 +1,2 @@
     Generated file for SomeProject.git
    +An important sentence for the README!
    """
When I run "dfetch update"
Then the output shows
    """
    Dfetch (0.13.0)
      SomeProject:
      > Fetched master - f47d80c35e14dfa4f9c9c30c9865cbf0f8d50933
      > Applying patch "SomeProject.patch"
        file 1/1:    b'README.md'
         hunk no.1 doesn't match source file at line 1
          expected: b'Generated file for SomeProject.git'
          actual  : b'Generated file for SomeProject'
        successfully patched 1/1:    b'README.md'
    """
And the patched 'MyProject/SomeProject/README.md' is
    """
    Generated file for SomeProject
    An important sentence for the README!
    This repository is used for testing.
    """

Schema

Below an overview of all possible fields on the manifest. The bold items are mandatory.

Note

A string should be a regular string without NULL or control characters.

type

object

properties

  • manifest

Top-level manifest object. See Manifest for details.

type

object

properties

  • version

type

string

  • remotes

List of remote sources. See Remotes for details. Each remote must be unique by its name.

type

array

items

type

object

properties

  • name

A unique name for the remote.

type

string

  • url-base

type

string

  • default

type

boolean

  • projects

List of projects to Dfetch. See Projects for details. Each project must be unique by its name.

type

array

items

type

object

properties

  • name

A unique name for the project.

type

string

  • dst

Destination path to fetch the project to, see Destination for details.

type

string

  • branch

type

string

  • tag

type

string

  • revision

type

string

  • url

type

string

  • repo-path

Path within the repository to fetch, see Repo-path for details.

type

string

  • remote

Name of remote to use as base, see Remotes for details.

type

string

  • patch

One or more patches to apply after fetching, see Patch.

type

string

  • vcs

Version control system used by the project. See VCS type for details.

type

string

enum

git, svn, archive

  • src

Source path within the repository to fetch, see Source for details.

type

string

  • ignore

Files to ignore. See Ignore for details.

type

array

items

type

string

  • integrity

Integrity verification block for archive dependencies. Only used with vcs: archive. Designed for future extension with sig: (detached signature URL) and sig-key: (signing-key URL or fingerprint) fields alongside hash:. See Archive for details.

type

object

properties

  • hash

Cryptographic hash of the archive file. Format: <algorithm>:<hex-digest>. Supported algorithms: sha256 (64 hex chars), sha384 (96 hex chars), and sha512 (128 hex chars). Example: sha256:e3b0c4….

type

string