A year ago I’ve built a setup where I can edit my Google drive documents with Vim but it didn’t catch because I was using drive which requires manual pulling and pushing, just like Git. So while I’m using Git on daily basis, Google drive is only needed when I want to collaborate or easily share a document which apparently is not so often. After short dry time and maybe some small issues, I forgot about it. However it’s still a sour point for me because I’m browsing with Qutebrowser and editing Google docs in it is not so fluent.

Yesterday, my friend Shaked shared with me a nice gem, GCSF:

GCSF is a virtual filesystem that allows users to mount their Google Drive account locally and interact with it as a regular disk partition.

GCSF is libfuse based, and there were other similar projects before but they seemed outdated when I wanted to try them. This is a very good news because now I don’t need to think about pulling or pushing. I just need to go to my local folder and edit the file.

How to edit a Google doc file locally with Vim:

1. Install GCSF and mount your Google drive.
2. Install Pandoc.
3. Browse to Google drive settings, check the box ‘Convert Uploads’.

4. Add to your Vim configuration (.vimrc):

   1 2	autocmd BufReadPost *.odt :%!pandoc -f odt -t markdown autocmd BufWritePost *.odt :!pandoc -f markdown -t odt % -o "%:r".odt

5. Create user service at ~/.config/systemd/user/:

6. Enable and start the service

   1 2	systemctl --user enable gcsf systemctl --user start gcsf

Notes:

• When you create a Google doc, it should have ‘odt’ extension in your local mount.
• If you want to create a Google doc locally, make sure it has ‘.odt’ suffix.
• You can add more file types in the same matter, such as ‘docx’ but it has little use in Google drive.
• Cooperative editing won’t work, and I won’t bet it will happen anytime soon or in this matter.
• Don’t edit it while others might be editing it.
• Pandoc conversion will destroy some WYSIWYG, for example pictures positions and fonts. Work with simple documents.

What is it good for?

• GCSF in general makes it easier to share with others, makes it more like Dropbox.
• Google drive allows hosting static webpages¹, maybe it can work with Hexo and the like
• Easy backup for documents and the like (not sure how good it will work)
• Initial drafting - Google docs has ‘suggestion’ and allows commenting which is more important in document development and you loose that in conversion. But the first step is the hardest step.

TL;DR

This post will explain how to trigger a job on Jenkins that will test changes on Github repository when Jenkins is behind a firewall. Github has service integration with Amazon SQS and Jenkins has a plugin that triggers a job on SQS event. To make it work you will need to do the following:

1. Create Amazon SQS for your Github repository and configure it
2. Add SQS to Github and configure it to notify on push and pull requests
3. Install SQS plugin and configure it in Jenkins
4. Write pipeline job that will handle the different events
5. Post job results to Github branch or pull request

Amazon SQS

“Amazon Simple Queue Service (SQS) is a fully managed message queuing service”, in other words SQS it will be our message bus, where we send messages from Github and listen to them on Jenkins. Jenkins will have to connect to SQS (outbond connection) and not listen to inbound connections. To get started, you need an AWS account and then you can create a new SQS service like this:

1. Go to AWS SQS console
2. Click on Create New Queue, and then:
   • What do you want to name your queue? Repository name¹
   • What type of queue do you need?² Standard Queue
3. Click on Quick-Create-Queue³
4. Select the queue you’ve just created and at the bottom copy for later:
   • ARN
   • URL

Next we will create a new IAM user for Jenkins:

1. Go to IAM console
2. Click on Add user, and then:
   • User name: jenkins⁴
   • Access type: Programmatic access
3. Click on Next:Permissions, and then:
   1. Select Attach existing policies directly
   2. In the Filter search for AmazonSQSFullAccess
4. Click on Next:Preview and then Click on Create user
5. Copy the Access Key ID and Secret access key⁵

That’s it, you have SQS configured.

Github

Github has a great support for webhooks and services, and there is even a one for Jenkins. But all of them assume Github can reach your Jenkins. Fortunately Github also has a service for Amazon SQS:

1. Go to your repository services: https://github.com/<user>/<repo>/settings/installations
2. Click on Add service, then find and select Amazon SQS
3. Click on Amazon SQS service to configure it, and then:
   • Aws access key - IAM user Access Key ID
   • Aws sqs arn - SQS queue ARN
   • Aws secret key - IAM user Secret access key
4. Update service

5. Open your terminal:

   1. Find the service id: curl -u <user> https://api.github.com/repos/<user>/<repo>/hooks

   2. Modify on which events the service will trigger: curl -X PATCH --data '{ "events": ["push", "pull_request"] }' -u <user> 'https://api.github.com/repos/<user>/<root>/hooks/<id>'

Github will now send messages on push and pull request to your Amazon SQS queue. You can find what other events can be added and the messages content at Github documentation on web-hooks and events.

Jenkins

Jenkins has a vast collection of open source plugins and even two for Amazon SQS. You will need only one of them and a pipeline support:

1. Go to Jenkins Plugin manager: http://<jenkins FQDN>/pluginManager/available
2. Install AWS SQS Build Trigger Plugin⁶
3. Go to Jenkins configuration: http://<jenkins FQDN>/configure

4. In section Configuration of Amazon SQS queues click on Add, then:

   1. Credentials: Click on Add and select Jenkins, then:
      • Kind: Secret Text
      • Scope: Global
      • Secret: IAM user’s Secret access key
      • ID: IAM user’s Access Key ID
   2. Queue name: Queue URL
   3. Click on Test access, you should see: “Access to <queue> successful”

5. Click on Save

Create a new Pipeline job⁷ with the following setting:

• In section Build Triggers:
  • mark: ‘Trigger build when a message is published to an Amazon SQS queue’
  • SQS queue to monitor: <queue>
• In section Pipeline, use Pipeline script and you can use this as template for your job:
  Pipeline
  • Lines 11-36 function for commenting on Github:
    • Install python library PyGithub on the executing system
    • Create Access token in Github for your user (or a bot user)
    • Set global parameter in Jenkins GITHUB_ACCESS_TOKEN
  • Lines 38-61 function for cloning and checking out the right repository based on the SQS message from Github.
  • Lines 69-73 will handle parameters from SQS trigger, the most important is sqs_body
  • Lines 76-87 will prepare everything you need for the job to work