Git integration with direktivctl

Modified on Tue, 23 May, 2023 at 12:37 AM

TABLE OF CONTENTS

Overview
direktivctl Integration

Overview

Documentation update:

https://docs.direktiv.io/environment/cli

Direktiv supports different mechanisms for Git authentication. The most common one is using access tokens, but SSH keys are also supported. When a new namespace or directory is created it can be connected to a Git repository.

access token

The Git integration between Direktiv and a Git repository is always one-way. Direktiv will synchronise the repository from the Git provider. The frequency or method of synchronisation is up to the user:

Scheduled: using the "Cron" feature described above
Action driven: from GitLab or GitHub
Command line driven: using direktiv-sync

This uses a simple cron job configuration. The structure of a cron configuration is shown below:

Here are a couple of examples:

Sample schedule	Cron job format
Every minute	* * * * *
Every Saturday at 23:45 (11:45 PM)	45 23 * * 6
Every Monday at 09:00 (9:00 AM)	0 9 * * 1
Every Sunday at 04:05 (4:05 AM)	5 4 * * SUN
Every weekday (Mon-Fri) at 22:00 (10:00 PM)	0 22 * * 1-5

direktivctl Integration

Sometimes direct integration between the Git repository and Direktiv isn't possible (as an example, you don't want to use GitLab runners in CI/CD pipelines to run an action).

Direktiv provides a command line tool to support command execution from a developers desktop (or IDE platform).

The direktiv-sync utility is available for download from the Direktiv GitHub releases page. The latest releases are published here:

https://github.com/direktiv/direktiv/releases

github.com_direktiv_direktiv_releases_tag_v0.7.1-rc1

To use the direktiv-sync utility, simply download the appropriate version for your desktop / laptop (in this case we will be using the direktiv_sync_darwin_arm64.tar.gz)

direktiv_sync_amd64.tar.gz for all Linux platforms
direktiv_sync_darwin.tar.gz for Intel based MacOS platforms
direktiv_sync_darwin_arm64.tar.gz for Apple Silicon based MacOS platforms
direktiv_sync_windows.tar.gz for Windows based platforms

Once the utility has been downloaded, simply unzip / unpack the utility into a folder which is in your device's execution path.

Create a Direktiv authentication token

Since we're configuring the direktiv-sync utility to access the Direktiv instance directly, authentication is needed to the Direktiv instance.

If the Open Source version has been deployed, then the authentication token is the "apikey" value configured during the installation (it's also used to authenticate when logging into the frontend).

The create an authentication token for the Enterprise Edition, the following steps can be followed:

Make sure you have logged into Direktiv and have access to the namespace you want to synchronise.
Click on the "Permissions" option in the left-hand menu. This will load the Group Policy and Policy Editor options.
In the top-right corner of the "Group Policy" window, click on the "+Auth Token" option.
The following options are needed to configure the Authentication Token:
1. Lifetime: the duration for which this token should be valid in ISO861 format. Example, a token that needs to last for 1 year would be "P1Y". To represent a duration of "three years, six months, four days, twelve hours, thirty minutes, and five seconds": "P3Y6M4DT12H30M5S"
2. Token description: describe the use for the token, as it will be displayed as a pop-up when hovering over the token once created
3. Permissions: select the permissions to allocate to the token. For Git integration the minimal permissions should explorerManage & explorerView.

NOTE: The token is generated and copied to the clipboard. Please take note of the token as it is not recoverable once created.

Configuration

Working with this tool assumes that you create a directory which is mirroring a namespace in Direktiv. This directory should be empty at start. The first thing to setup is the connectivity to Direktiv. For this a .direktiv.yaml file has to be created within this directory.

This file needs the Authentication Token (previously created), the address of the Direktiv instance and the namespace it should use.

auth: "eyJhbGciOiJIUzIT-FEeC-nzAp__bWq74"
addr: "https://dev.direktiv.io"
namespace: "direktiv-sync-namespace"

Let's assume we have the following structure in our IDE of choice (in this case Visual Studio Code):

And the Direktiv platform is configured with a namespace only:

Synchronizing the Direktiv instance and local directory

The following command will execute a push of all the local files to the Direktiv instance:

# direktivctl workflows push .
Found 1 Local Workflow/s to update
[1/1] Updating Namespace: 'direktiv-sync-namespace' Workflow: 'sample-workflow'
updating namespace: 'direktiv-sync-namespace' workflow: 'sample-workflow'
Found 2 Local Variables to push to remote
      Updating Remote Workflow Variable: 'py-tweets.py'
      Updating Remote Workflow Variable: 'requirements.txt'
      Successfully updated remote workflow

And the result in the Direktiv namespace called direktiv-sync-namespace is:

There are also other options available. To synchronize a subdirectory, the command below can be used (or a specific file):

% direktivctl workflows push subdir 
Found 1 Local Workflow/s to update
[1/1] Updating Namespace: 'direktiv-sync-namespace' Workflow: 'subdir/sample-subdir-workflow'
updating namespace: 'direktiv-sync-namespace' workflow: 'subdir/sample-subdir-workflow'
Successfully updated remote workflow

Executing a workflow

Users of the sync utility can also execute a workflow remotely. Using the example above, the workflow expects an JSON input to the workflow when executing. The JSON object is shown below:

{
"bearer_token":"AAAAAAAAAiHfUH2bGgM5cNEUdeAYjFU4iz8oGT0e38VKyVjhYlNYLrhlZ6F",
   "twitter_searchstring":"direktiv",
   "max_search_returns":10
}

The workflow can be executed as follow:

direktivctl workflows  exec sample-workflow.yaml

NOTE: The exec command uploads the latest version of the workflow and executes the it. During execution the logs are printed to stdout.

An example of the workflow execution is shown below:

# direktivctl workflows exec sample-workflow.yaml
updating namespace: 'direktiv-sync-namespace' workflow: 'sample-workflow'
Found 3 Local Variables to push to remote
Updating Remote Workflow Variable: 'input.json'
Updating Remote Workflow Variable: 'py-tweets.py'
Updating Remote Workflow Variable: 'requirements.txt'
Successfully Executed Instance: c48a1c49-0b92-4677-94d8-89e7278f3148
-------INSTANCE LOGS-------
06 Feb 23 16:52 PST: Preparing workflow triggered by api.
06 Feb 23 16:52 PST: Running state logic (step:1) -- getter
06 Feb 23 16:52 PST: Transforming state data.
06 Feb 23 16:52 PST: Transitioning to next state: run-code (2).
06 Feb 23 16:52 PST: Running state logic (step:2) -- run-code
06 Feb 23 16:52 PST: Sleeping until child '134132ab-d848-4b31-b163-ec78ee14b4d0' returns (python).
06 Feb 23 16:52 PST: running command /root/.pyenv/bin/pyenv local 3.8.13

06 Feb 23 16:52 PST: running command /root/.pyenv/shims/python3 -m pip install -r requirements.txt

06 Feb 23 16:52 PST: Collecting tweepy

06 Feb 23 16:52 PST:   Downloading tweepy-4.12.1-py3-none-any.whl (101 kB)

06 Feb 23 16:52 PST:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 101.6/101.6 KB 1.3 MB/s eta 0:00:00
06 Feb 23 16:52 PST: 

06 Feb 23 16:52 PST: Collecting requests<3,>=2.27.0

06 Feb 23 16:52 PST:   Downloading requests-2.28.2-py3-none-any.whl (62 kB)

06 Feb 23 16:52 PST:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 KB 664.3 kB/s eta 0:00:00
06 Feb 23 16:52 PST: 

06 Feb 23 16:52 PST: Collecting requests-oauthlib<2,>=1.2.0

06 Feb 23 16:52 PST:   Downloading requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)

06 Feb 23 16:52 PST: Collecting oauthlib<4,>=3.2.0

06 Feb 23 16:52 PST:   Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)

06 Feb 23 16:52 PST:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 151.7/151.7 KB 1.7 MB/s eta 0:00:00
06 Feb 23 16:52 PST: 

06 Feb 23 16:52 PST: Collecting urllib3<1.27,>=1.21.1

06 Feb 23 16:52 PST:   Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)

06 Feb 23 16:52 PST:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 KB 56.0 MB/s eta 0:00:00
06 Feb 23 16:52 PST: 

06 Feb 23 16:52 PST: Collecting charset-normalizer<4,>=2

06 Feb 23 16:52 PST:   Downloading charset_normalizer-3.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (195 kB)

06 Feb 23 16:52 PST:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 195.4/195.4 KB 17.2 MB/s eta 0:00:00
06 Feb 23 16:52 PST: 

06 Feb 23 16:52 PST: Collecting idna<4,>=2.5

06 Feb 23 16:52 PST:   Downloading idna-3.4-py3-none-any.whl (61 kB)

06 Feb 23 16:52 PST:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 KB 8.5 MB/s eta 0:00:00
06 Feb 23 16:52 PST: 

06 Feb 23 16:52 PST: Collecting certifi>=2017.4.17

06 Feb 23 16:52 PST:   Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)

06 Feb 23 16:52 PST:      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 KB 1.7 MB/s eta 0:00:00
06 Feb 23 16:52 PST: 

06 Feb 23 16:52 PST: Installing collected packages: charset-normalizer, urllib3, oauthlib, idna, certifi, requests, requests-oauthlib, tweepy

06 Feb 23 16:52 PST: Successfully installed certifi-2022.12.7 charset-normalizer-3.0.1 idna-3.4 oauthlib-3.2.2 requests-2.28.2 requests-oauthlib-1.3.1 tweepy-4.12.1 urllib3-1.26.14

06 Feb 23 16:52 PST: WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

06 Feb 23 16:52 PST: WARNING: You are using pip version 22.0.4; however, version 23.0 is available.
You should consider upgrading via the '/root/.pyenv/versions/3.8.13/bin/python3 -m pip install --upgrade pip' command.

06 Feb 23 16:52 PST: running command /root/.pyenv/shims/python3 py-tweets.py -b AAAAAAAAAAAAAAAAAAAAAAg2fwEAAAAAdO8ZWozFdJnpfAfcOSdUvIm%2FcRY%3DiHfUH2bGgM5cNEUdeAYjFU4iz8oGT0e38VKyVjhYlNYLrhlZ6F -t direktiv -m 10

06 Feb 23 16:53 PST: [
   {
      "tweet_id": 1622717760802357260,
      "tweet_text": "@otkaznik @AngelicaJkpg @wedholmen Det bygger på ett EU direktiv kallat GDPR som vi måste följa oavsett regering. Eftersom vi gett mer makt till EU än vad som var tänkt får folk skylla sig själva om de blir kränkta."
   },
   {
      "tweet_id": 1622717133405782035,
      "tweet_text": "@TorbjornDokken @AngelicaJkpg Röstade du för EU har du fått vad du gnäller över. Det bygger på ett EU direktiv."
   },
   {
      "tweet_id": 1622709877498937370,
      "tweet_text": "RT @Ericson_ubbhult: Lögnblaskan @aftonbladet levererar. Regeringen har varit tydlig - man vill ha skarpare (!) direktiv till korruptionsut…"
   },
   {
      "tweet_id": 1622648456685682718,
      "tweet_text": "RT @TVermes: Joda,  EU-folkene banet vei. Statsminister Gro sendte i 1988 ut direktiv om at nye norske lover skulle harmoniseres med EUs lo…"
   },
   {
      "tweet_id": 1622632360536580098,
      "tweet_text": "RT @TVermes: Joda,  EU-folkene banet vei. Statsminister Gro sendte i 1988 ut direktiv om at nye norske lover skulle harmoniseres med EUs lo…"
   },
   {
      "tweet_id": 1622630899186532355,
      "tweet_text": "Joda,  EU-folkene banet vei. Statsminister Gro sendte i 1988 ut direktiv om at nye norske lover skulle harmoniseres med EUs lover - 4 år før #EØS-avtalen ble inngått. https://t.co/8rxDKTT9Jm"
   },
   {
      "tweet_id": 1622627316525408256,
      "tweet_text": "@peterwennblad Det är något konstigt med LKAB:s Vd. Eller så har det kommit politiska direktiv."
   },
   {
      "tweet_id": 1622617392718266370,
      "tweet_text": "@PJiborn @snyggbonde @SkogAndersP Det är väl för Norra Sve som Sve tolk av EUs direktiv som får störst betydelse med krav på skydd av jättearealer taiga, ändras detta så är det väl inge problem att skydda mer löv i Söder om behovet finns."
   },
   {
      "tweet_id": 1622606578200313857,
      "tweet_text": "Looking for digital transformation? You need #serverlessarchitecture. Users have seen:\n\n- 70% improvement in time to market ⏱\n- 60% increase in developer productivity ?‍?\n- 18% - 31% decrease in infrastructure, integration and development costs ?\n\nhttps://t.co/VsaRTi5p2h https://t.co/bKpSkYM2lN"
   },
   {
      "tweet_id": 1622581251193417731,
      "tweet_text": "Åh... ännu mer dumheter... ett direktiv som skyddar förövarna och straffar de som försöker skydda sig själva.\n\nhttps://t.co/v38qGereCx"
   }
]

06 Feb 23 16:53 PST: Child '134132ab-d848-4b31-b163-ec78ee14b4d0' returned.
06 Feb 23 16:53 PST: Transforming state data.
06 Feb 23 16:53 PST: Transitioning to next state: run-shell (3).
06 Feb 23 16:53 PST: Running state logic (step:3) -- run-shell
06 Feb 23 16:53 PST: Sleeping until child '62cdcbcf-6d73-47b8-82c8-40750b993f6f' returns (shell).
06 Feb 23 16:53 PST: running command /usr/bin/echo 1622709877498937300

06 Feb 23 16:53 PST: 1622709877498937300

06 Feb 23 16:53 PST: Child '62cdcbcf-6d73-47b8-82c8-40750b993f6f' returned.
06 Feb 23 16:53 PST: Workflow completed.
Instance Completed With Status: complete

------INSTANCE OUTPUT------
{
  "result": [
    {
      "tweet_id": 1622717760802357200,
      "tweet_text": "@otkaznik @AngelicaJkpg @wedholmen Det bygger på ett EU direktiv kallat GDPR som vi måste följa oavsett regering. Eftersom vi gett mer makt till EU än vad som var tänkt får folk skylla sig själva om de blir kränkta."
    },
    {
      "tweet_id": 1622717133405782000,
      "tweet_text": "@TorbjornDokken @AngelicaJkpg Röstade du för EU har du fått vad du gnäller över. Det bygger på ett EU direktiv."
    },
    {
      "tweet_id": 1622709877498937300,
      "tweet_text": "RT @Ericson_ubbhult: Lögnblaskan @aftonbladet levererar. Regeringen har varit tydlig - man vill ha skarpare (!) direktiv till korruptionsut…"
    },
    {
      "tweet_id": 1622648456685682700,
      "tweet_text": "RT @TVermes: Joda,  EU-folkene banet vei. Statsminister Gro sendte i 1988 ut direktiv om at nye norske lover skulle harmoniseres med EUs lo…"
    },
    {
      "tweet_id": 1622632360536580000,
      "tweet_text": "RT @TVermes: Joda,  EU-folkene banet vei. Statsminister Gro sendte i 1988 ut direktiv om at nye norske lover skulle harmoniseres med EUs lo…"
    },
    {
      "tweet_id": 1622630899186532400,
      "tweet_text": "Joda,  EU-folkene banet vei. Statsminister Gro sendte i 1988 ut direktiv om at nye norske lover skulle harmoniseres med EUs lover - 4 år før #EØS-avtalen ble inngått. https://t.co/8rxDKTT9Jm"
    },
    {
      "tweet_id": 1622627316525408300,
      "tweet_text": "@peterwennblad Det är något konstigt med LKAB:s Vd. Eller så har det kommit politiska direktiv."
    },
    {
      "tweet_id": 1622617392718266400,
      "tweet_text": "@PJiborn @snyggbonde @SkogAndersP Det är väl för Norra Sve som Sve tolk av EUs direktiv som får störst betydelse med krav på skydd av jättearealer taiga, ändras detta så är det väl inge problem att skydda mer löv i Söder om behovet finns."
    },
    {
      "tweet_id": 1622606578200313900,
      "tweet_text": "Looking for digital transformation? You need #serverlessarchitecture. Users have seen:\n\n- 70% improvement in time to market ⏱\n- 60% increase in developer productivity ?‍?\n- 18% - 31% decrease in infrastructure, integration and development costs ?\n\nhttps://t.co/VsaRTi5p2h https://t.co/bKpSkYM2lN"
    },
    {
      "tweet_id": 1622581251193417700,
      "tweet_text": "Åh... ännu mer dumheter... ett direktiv som skyddar förövarna och straffar de som försöker skydda sig själva.\n\nhttps://t.co/v38qGereCx"
    }
  ],
  "return": {
    "bash": [
      {
        "result": 1622709877498937300,
        "success": true
      }
    ]
  },
  "success": true
}