Dependencies

All stuff you need to use during development process are coming together with Control Server installation in onboard nexus:

Nexus URL

https://<CS URL>/nexus/#browse/browse:rpaplatform

Project generation

Project can be generated from archetype using either CLI or IDE.

CLI

The following command creates a project from archetype:

mvn archetype:generate -DarchetypeGroupId=eu.ibagroup -DarchetypeArtifactId=easy-rpa-ap-archetype -DarchetypeVersion=2.4.0-SNAPSHOT -DgroupId=org.test -DartifactId=sample -Dversion=1.0-SNAPSHOT

As a result the stub project for org.test:1.0-SNAPSHOT will be generated

IDE

First, add remote catalog with archetype easy-rpa-ap-archetype in your IDE

Second, use the sequence "New project → Maven project", then choose easy-rpa-ap-archetype as the base archetype to generate project from.

Project structure

The generated project for abstract Automation Process is usual maven project with additional folder:

The folder package contains definition of artifacts that need to be added into AutomationProcess package - the result of maven project deploy goal. That you can use the generated package to import into a EasyRPA Control Server.

Folder

Content

Definition

automationProcess

(or ap for old releases)

Automation Processes to add into result package.

Contains a set of the following files per Automation process, the files with the same AP should have the same name, but different extensions.

file

description

definition

*.json

required

eu.ibagroup.easyrpa.cs.controller.dto.AutomationProcessImportDto - JSON

Here is an usual definition template:

{
	"importStrategy": "OVERRIDE",
	"jarImportStrategy": "SKIP",
	"name": "Sample AP",
	"description": "Sample AP",
	"repositoryId": "test.org:sample:jar:0.1-SNAPSHOT",
	"moduleClass": "com.ap.Module",
	"input": {
		"variables": {}
	}
}

where:

importStrategy - ADD, OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, ADD if skipped.
jarIportStrategy - OVERRIDE, SKIP. The merge strategy for AP libraries in automatic merge. Optional, if skipped, auto-merge is not allowed. (*)
moduleClass - the AP class name, required
name - Automation process name, required, You can skip it, if Automation Process's @ApModuleEntry annotation has name, the plugin sets it automatically.
description - Automation process description, optional, You can skip it, if Automation Process's @ApModuleEntry annotation has description, the plugin sets it automatically.
repositoryId - Ap artifacts id to download from nexus, required. You can skip it, the plugin sets it from current maven project automatically. For AP, defined in different libs, you have to set it manually.
input - defines json of AP input. Only variable field is acceptable. Optional.

*.properties

optional

property file for Automation Process parameters of the corresponding *.json. The parameters could be defined in JSON file, but this file adds and overrides defined in json.

key=value

* For every defined Automation Process, export plugin try to add a corresponding jar into result package lib folder with AP class definition. If all AP in the same project, will be only one jar. If you want to add additional jar(s) into package you can specify what to add in the projects pom in pluginin section:

<build>
	<plugins>
		<plugin>
			<groupId>eu.ibagroup</groupId>
			<artifactId>easy-rpa-ap-export-plugin</artifactId>
			<configuration>
				<groupId>${project.groupId}|eu.ibagroup|eu.ibagroup.samples.system|eu.ibagroup.samples</groupId>
				<artifactId>easy-rpa-dp-ap|easy-rpa-cldp-ap|easy-rpa-iedp-ap|easy-rpa-aps|easy-rpa-idp-ap|easy-rpa-invoiceplane-system|easy-rpa-invoiceplane</artifactId>
			</configuration>
		</plugin>
	</plugins>
</build>

where groupId/artifactId are regexp string. The plugin add additional libs from project dependencies scope of all artifact that has corresponding groupId/artifactId

dataStore

(or datastore for old releases)

Data Stores to add into result package.

Contains a set of the following files per DataStore, the files with the same DS should have the same name, but different extensions.

file

description

definition

*.json

required

eu.ibagroup.easyrpa.cs.controller.dto.DataStoreImportDto - JSON

Here is an usual definition template:

{
	"importStrategy": "OVERRIDE",
	"description": "Sample",
	"name": "SAMPLE_DS",
	"schema": {
		"columns": [
			"name",
			"author"
		],
		"indexes": [
			{
				"columns": [
					{
						"name": "author",
						"isAscending": false
					}
				],
				"unique": true
			}
		]
	}
} }

Where:

importStrategy - ADD, OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, ADD if skipped.
name - the datastore name, required
description - a description, optional
schema - the datastore definition with columns and indexes, optional

*.csv

optional

the datastore content

Csv file format:

index,name,author
1,Book 1,Author 1
2,Book 2,Author 2

*.format

optional

the csv file format, the JSON of eu.ibagroup.easyrpa.cs.controller.dto.TableDataFormatDto

Here is an sample of custom format template:

{
	"charset": "UTF-8",
	"delimiter": ",",
	"ignoreEmptyLines": true,
	"ignoreSurroundingSpaces": false,
	"nullString": null,
	"quote": """,
	"recordSeparator": "
"
}

documentSet

(or docset for old releases)

Document Sets to add into result package.

Contains a set of the following files per DocumentSet, the files with the same DS should have the same name, but different extensions.

file

description

definition

*.json

required

eu.ibagroup.easyrpa.cs.controller.dto.DocumentSetImportDto - JSON

Here is an usual definition template:

{
 "importStrategy": "OVERRIDE",
 "name": "SAMPLE_DS",
 "description": "Sample Document Set",
 "documentType": {
	 "name": "Sample Document"
 },
 "model": {
	 "name": "idp_sample_invoice",
	 "version": "1.0.11"
	},
 "settings": {} 
 "docSetSchema" : {
	"schema" : {
		"columns" : ["document_type", "model_document_type", "document_type_score", "cl_result", "ie_result", "error_message" ],
		"indexes" : [{
				"columns": [
					{
						"name": "document_type",
						"isAscending": false
					}
				],
				"unique": false
			}]
	},
	"metadata" : [ ]
	}
}

Where:

importStrategy - OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, if skipped, auto-merge is not allowed.
name - the document set name, required
description - a description, optional
documentType - the document set's document type reference by name, required
model - the model reference, optional
settings - settings JSON
docSetSchema - the datastore definition with custom columns and indexes, optional

*.zip

optional

Document Set data archive file - the document Set content.

Use Document Set data export option to obtain a file.

documentType

(or dt for old releases)

Document Types to add into result package.

Contains one file per DocumentType.

file

description

definition

*.json

required

eu.ibagroup.easyrpa.cs.controller.dto.DocumentTypeImportDto - JSON

Here is an usual definition template:

{
	"importStrategy": "OVERRIDE",
	"name": "SAMPLE_DT",
	"description": "Sample Document Type",
	"humanTaskTypeName": "Basic Sample Type",
	"settings": {}
}

Where:

importStrategy - ADD, OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, ADD if skipped.
name - the document set name, required
description - a description, optional
humanTaskTypeName- the human task type reference by name, required
settings - settings JSON

model

Models to add into result package.

Contains a set of the following files per Model, the files with the same model should have the same name, but different extensions.

file

description

definition

<name-version>.tar.json

optional

eu.ibagroup.easyrpa.cs.controller.dto.MlModelImportDto - JSON

Here is an usual definition template:

{
	"importStrategy": "OVERRIDE",
	"description": "Sample Model"
}

Where:

importStrategy - OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, if skipped, auto-merge is not allowed.
description - a description, optional

If the file is not present, the system import model using gz file without description and automatic merge strategy, you have to specify it manually in UI, and can not perform automatic import in export plugin

<name-version>.tar.gz

required

Python model package file.

Use MlModel export option to obtain a file.

humanTaskType

(or htt for old releases)

Human Task Type to add into result package.

Contains a set of the following files per Human Task Type, the files with the same HTT should have the same name, but different extensions.

file

description

definition

*.json

required

eu.ibagroup.easyrpa.cs.controller.dto.HumanTaskTypeImportDto - JSON

Here is an usual definition template:

{
	"importStrategy": "OVERRIDE",
	"name": "SAMPLE_DT",
	"description": "Sample Document Type"
}

Where:

importStrategy - ADD, OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, ADD if skipped.
name - the document set name, required
description - a description, optional

*.zip

required

Human Task Type package file.

The Human Task Type editor zip archive.

secretVault

(or secret for old releases)

Secret Vault entries to add into result package.

Contains one vault.properties file for all secrets or one file per Secret entry. In case if the vault.properties exists, the json files will be skipped.

file

description

definition

vault.properties

optional

Contains secrets definition in property file format.

Csv file format:

importStrategy=OVERRIDE
abcLogin={"user": "user1", "password": "Password1"}
key=value

Where:

importStrategy - the key is reserved - OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, if skipped, auto-merge is not allowed.
value could be the eu.ibagroup.easyrpa.engine.model.SecretCredentials json in case to use secret as credentials

*.json

optional

eu.ibagroup.easyrpa.cs.controller.dto.SecretVaultImportDto - JSON

Here is an usual definition template:

{
	"importStrategy": "OVERRIDE",
	"alias": "key",
	"value": "value"
}

Where:

importStrategy - OVERRIDE, SKIP. The merge strategy for automatic merge. Optional, if skipped, auto-merge is not allowed.
alias - the secret aliace, required
value - the secret value, required

The Human Task Type editor zip archive.

storage

Files to add into EasyRPA storage.

The directory should contains at least one level folder that is mapped to a bucket. i.e. the following structure will add all pdf files into folder idp_sample of the bucket data:

package.json

Automation Package format mark file.

Empty JSON

src/main/resources/usecase.properties

Data Store name runtime rendering pattern and default use case properties that relates to a specific Use Case implemented

Mandatory 'uc_resource_name_pattern' property:

uc_resource_name_pattern=UC_${uc_name}_${entity_name}_${uc_version}

Optional default properties of use case:

uc_name=SMPL
uc_version=1

(warning) Please, note, that only OVERRIDE and SKIP strategies can be applied to .jar, document set, model and secret vault entities. Be careful when using the OVERRIDE strategy for the previously mentioned entities. Even if an error occurs during the import process, the changes cannot be rolled back and the data in the bucket will be lost for the .jar, document set, model and secret vault if the OVERRIDE strategy was applied to them.

Package building

The following command will build the project and create a zip-package with all needed files including jar.

mvn clean package

The zip file of project will be generated and put into target directory

Package importing

There are three ways to import package into control server:

Deploy on Control Server manually
Upload AP zip package
AP Export maven plugin

Deploy on Control Server manually

Deploy AP jar on easyRPA nexus.

Set easy-rpa-nexus repository credentials in your maven settings.xml, see above.

Set-up nexus repository URL in distributionManagement section of your project pom file:

<distributionManagement>
	<repository>
		<id>easy-rpa-nexus</id>
		<url>https://<easy-rpa-host>/nexus/repository/aps/</url>
	</repository>
	<snapshotRepository>
		<id>easy-rpa-nexus</id>
		<url>https://<easy-rpa-host>/nexus/repository/aps/</url>
	</snapshotRepository>
</distributionManagement>

Run maven deploy:

mvn clean deploy

Then go to control server and define your AP, Secrets, Datastores:

Upload AP zip package with Contrlol Server UI

Navigate to page Automation Process → Upload package
Choose the zip file to upload
Click Upload.

Control server will respond with report about operations it has done and their status. UI for package upload is using ADD mode by default which results in creation of records which did not exist before and fail in opposite case.

There are two possible scenarios:

1) There are no errors, and all components have been successfully created (no further actions required)

2) There are errors - some components failed due to conflicts with existing ones with the same name. In this case review the errors and choose different strategy to merge this components, such as OVERRIDE or SKIP. Press Upload button to perform upload operation once again.

Note, that the upload operation internally consists from two independent transactions - CS update and upload of jar file to Nexus. Upload to Nexus is not rolled back due to nexus limitations.

AP Export with maven plugin

The cs-upload profile can be used for CI/CD builds. It contains maven-assembly-plugin that uploads the generated package to CS.

To use plugin for automatical Ap package upload on Control Server, you need to set path to development configuration file - cs.json:

<build>
	<plugins>
		<plugin>
			<groupId>eu.ibagroup</groupId>
			<artifactId>easy-rpa-ap-export-plugin</artifactId>
			<version>2.4.0-SNAPSHOT</version>
			<configuration>
				<config>c:\work\cs.json</config>
			</configuration>
		</plugin>
	</plugins>
</build>

And run the following command that will build the project, create a zip-package and upload it into CS the development configuration you specified:

mvn clean package deploy -Pcs-upload

Note, that plugin is using importStrategy defined in package for automatic conflict resolving.

AP Standalone Run

The standalone-jar maven profile (included into artefactory-generated pom by default) can be used for generating standalone self-executable jar will all needed dependencies and resources included. It contains maven-shade-plugin that resolve all AP dependencies and combines them into single executable Jar, plus it include apm_run.properties and vault.properties from test resources.

Below is profile snippet that can be included into an existing AP module pom.xml:

		 <profile>
			<!-- use this profile to generate standalone executable jar with the ap -->
			<id>standalone-jar</id>
			<dependencies>
				<dependency>
					<groupId>eu.ibagroup</groupId>
					<artifactId>easy-rpa-engine</artifactId>
					<scope>compile</scope>
				</dependency>
			</dependencies>
			<build>
				<plugins>
					<plugin>
						<groupId>org.apache.maven.plugins</groupId>
						<artifactId>maven-shade-plugin</artifactId>
						<executions>
							<execution>
								<goals>
									<goal>shade</goal>
								</goals>
								<configuration>
									<shadedArtifactAttached>true</shadedArtifactAttached>
									<shadedClassifierName>full</shadedClassifierName>
									<transformers>
										<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
											<mainClass>eu.ibagroup.easyrpa.engine.boot.ApModuleRunner</mainClass>
										</transformer>
										<transformer implementation="org.atteo.classindex.ClassIndexTransformer" />
										<transformer implementation="org.apache.maven.plugins.shade.resource.IncludeResourceTransformer">
											<resource>vault.properties</resource>
											<file>src\test\resources\vault.properties</file>
										</transformer>
										<transformer implementation="org.apache.maven.plugins.shade.resource.IncludeResourceTransformer">
											<resource>apm_run.properties</resource>
											<file> src\test\resources\apm_run.properties</file>
										</transformer>										
									</transformers>
									<filters>
										<filter>
											<artifact>eu.ibagroup:easy-rpa-engine-api</artifact>
											<excludes>
												<exclude>eu/ibagroup/easyrpa/engine/apflow/DevelopmentApModule.class</exclude>
											</excludes>
										</filter>
										<filter>
											<artifact>*:*</artifact>
											<excludes>
												<exclude>module-info.class</exclude>
												<exclude>META-INF/*.SF</exclude>
												<exclude>META-INF/*.DSA</exclude>
												<exclude>META-INF/*.RSA</exclude>
											</excludes>
										</filter>
									</filters>
								</configuration>
							</execution>
						</executions>
						<dependencies>
							<dependency>
								<groupId>org.atteo.classindex</groupId>
								<artifactId>classindex-transformer</artifactId>
								<version>3.13</version>
							</dependency>
						</dependencies>
					</plugin>
				</plugins>
			</build>
		</profile>

And run the following command that will build the project and create a standalone executable jar :

mvn clean package -Pstandalone-jar

Once uber jar generated it can be run with the command line like below:

java -jar sample-1.0-SNAPSHOT-standalone.jar

Please see Invoice Plane Automation Demo for an example.

Automation Process Project structure

Automation Process Project structure

Dependencies

Project generation

CLI

IDE

Project structure

Package building

Package importing

Deploy on Control Server manually

Upload AP zip package with Contrlol Server UI

AP Export with maven plugin

AP Standalone Run