Open topic with navigation

Set up a recurring task to spawn workers

To ensure that incoming Helix Core events are automatically processed by Swarm, it is important to set up a cron job to do this. The cron job can be installed on any host, although you may want to place this on the Swarm host.

The recurring task to invoke Swarm workers, installed in a later step, requires either of:

• curl
  https://curl.haxx.se/download.html

  Note

  For Windows, curl.exe depends on MSVCR100.dll. You can get a copy by installing the Microsoft Visual C++ Redistributable Package, available for:

  • 32-bit systems: https://www.microsoft.com/download/en/details.aspx?id=5555
  • 64-bit systems: https://www.microsoft.com/download/en/details.aspx?id=14632

  If you install Swarm with HTTPS, curl.exe requires recent CA certificates (or HTTPS connections silently fail). You can get a copy of the cacert.pem from:

  https://curl.haxx.se/docs/caextract.html

  Once downloaded, copy cacert.pem to the same folder where you installed curl.exe, and rename it to curl-ca-bundle.crt.

  Warning

  If curl (or curl.exe on Windows) cannot execute as expected, trigger execution may block or fail. For example, if MSVCR100.dll is missing from a Windows system, invoking curl.exe causes a dialog to appear.

  Prior to configuring the triggers, verify that curl executes. On Linux systems, run:

   $ curl -h

  On Windows systems, run:

  C:\> curl.exe -h

  The start of the output should be similar to:

  Usage: curl [options...] <url>
  Options: (H) means HTTP/HTTPS only, (F) means FTP only
       --anyauth       Pick "any" authentication method (H)
   -a, --append        Append to target file when uploading (F/SFTP)
       --cacert FILE   CA certificate to verify peer against (SSL)
       --capath DIR    CA directory to verify peer against (SSL)
  ...[truncated for brevity]...

  For a more thorough test that actually fetches content over a network, try the following test:

  • For Linux systems, run:

    $ curl https://www.perforce.com/

  • For Windows systems, run:

     C:\> curl.exe https://www.perforce.com/

  The output should look like HTML.

• wget
  https://ftp.gnu.org/gnu/wget/

  http://gnuwin32.sourceforge.net/packages/wget.htm (for Windows)

  Note

  If you are using Powershell on Windows systems, be aware that Powershell includes aliases for curl and wget that call the Powershell command Invoke-WebRequest instead of curl.exe or wget.exe. Invoke-WebRequest has different command-line options than either curl or wget, which can be confusing.

  If you want to remove the built-in aliases for curl and wget from Powershell, follow these steps:

  1. Create a Powershell profile (only if you have not already done so):

      PS C:\> New-Item $profile -force -itemtype file

  2. Edit your profile:

      PS C:\> notepad $profile

  3. Add the following line to your profile:

     remove-item alias:curl
     remove-item alias:wget

  4. Save the profile and close notepad.

  5. Reload your profile:

      PS C:\> . $profile

  Warning

  curl or wget must be installed or workers do not spawn and Swarm cannot process any events. See below for verification steps.

1. Create a file named helix-swarm in /etc/cron.d.
2. Edit /etc/cron.d/helix-swarm to contain one of the following blocks; select a block depending on whether your system has curl or wget installed.

   • If you have curl installed:

     # This ensures that a worker is fired up every minute
     * * * * * nobody curl -so /dev/null -m5 https://myswarm.url/queue/worker

   • If you have wget installed:

     # This ensures that a worker is fired up every minute
     * * * * * nobody wget -q -O /dev/null -T5 https://myswarm.url/queue/worker

3. Replace myswarm.url above with the actual URL you have configured for Swarm (which may include a sub-folder or a custom port).

   If the cron job is running on the Swarm host, and you have specified the correct hostname item in the Environment configuration, this can be set to localhost.

   In the example configuration lines above, where you see -m5 or -T5, the 5 is the number of seconds that the cron task will wait for a response from the Swarm host. When the cron task is installed on the Swarm host, such as in the Swarm OVA, that value could be reduced to 1 seconds (e.g. -m1 or -T1).

   Note

   If you configure Swarm to use HTTPS, and you install a self-signed certificate, the cron jobs need to be adjusted to avoid certificate validity test which could cause silent failures to process events.

   • If you have curl installed:

     # This ensures that a worker is fired up every minute
     * * * * * nobody curl -so /dev/null --insecure -m5 https://myswarm.url/queue/worker

   • If you have wget installed:

     # This ensures that a worker is fired up every minute
     * * * * * nobody wget -q -O /dev/null --no-check-certificate -T5 https://myswarm.url/queue/worker

4. Save the edited file.

You are now all set to start using Swarm. Enjoy!

Note

If the recurring task is disabled, or stops functioning for any reason, logged-in users see the following error message when Swarm detects that no workers are running:

curl/wget verification

The cron job depends on having curl or wget installed, as indicated in Runtime dependencies.

To verify that curl or wget is installed, use the which command. For example:

$ which curl

If you see any output, the referenced command is installed.