Rules are probably the most useful feature in Mergado. With queries, they provide a tool for fast and easy modification of products’ data.

Queries

Queries are used to filter products by values of their elements. For this purpose, we implemented our own language abbreviated as MQL (Mergado Query Language). MQL helps to describe how to filter a set of products, similarly as SELECT statements in SQL. And in fact, all MQLs are always translated into SQL. For example, a query returning all products with price lower than 100 EUR looks like this in MQL:

PRICE_VAT < 100

In the above query, PRICE_VAT is the name of an element in a project (which is often also the name in the project’s input XML feed, however, elements can also be created manually in Mergado UI by users or in the API). Anyway, it’s possible to create queries using a bit more advanced conditions. Here is a list of all possible operators:

• >, <, >=, <=, =, != – comparison of values
• ~, !~ – comparison by regular expression
• IN, NOT IN – comparison with a list of values (surrounded by parenthesis)
• OR, AND – logical operators

Names of elements can be escaped using square brackets, i.e. [ and ]. For example, the element PARAM|color can be used in MQL like this: [PARAM|color] = 'red'.

Rules

All manipulation of products’ data happens in rules. Rules transform input data (provided mostly by administrators in eshops or by marketing agencies) to output data (usually consumed by services like Heureka.cz, Google Merchant and Zboží.cz). The simplest rule is called rewriting which does nothing else than that it fills the output value of a product’s element with a value. For instance, imagine you have a product like ThinkPad T440 and you want to lower its price. The rule rewriting is one way to do it and this is the action it performs:

        input                      output

    +------------+             +------------+
    |  product   |             |  product   |
    +------------+             +------------+
    | NAME:      |             | NAME:      |
    |   ThinkPad |  rewriting  |   ThinkPad |
    |            | +---------> |            |
    | PRICE:     |             | PRICE:     |
    |   999 EUR  |             |   819 EUR  |
    +------------+             +------------+

You usually don’t want to lower prices of all products in your feed and that is where queries come into play as they are used to select only a portion of products in a feed.

Instantiation and definition

When we say instantiate a rule, what we mean is that someone creates a rule that performs bulk modification of products in a feed. This modification is done in a way that the user intents by an instantiation, for example, a rule’s instance might lower prices of all products by 100 EUR. Such rule is probably not very useful but it shows what rule instances do: modify a project’s (export’s) products.

Another term we use is rule definition. Rule definitions are actually like classes in object-oriented programming, while rule instances are like objects. A rule definition represents what a particular rule does and what parameters it requires in order to be instantiated.

For example, the definition of the rule rewriting looks like this:

{
    "type": "rewriting",
    "relationship": "1:1",
    "fields": [
        {
            "required": false,
            "type": "STRING",
            "name": "new_content"
        }
    ]
}

As we mentioned previously, this rule rewrites the current value with a new one usually chosen by the user (or by an app). When you want an app to instantiate a new rule using our API, you must provide the following information:

• type – The type of rule to instantiate.
• data – A list of objects in case of 1:N relationship or an object in case of 1:1 relationship. Each object represents a rule-specific data for the rule instantiation. The field fields defines the name, type and other information of each field of the object for instantiation.

In the case of the rewriting rule, a user or an app provides the new_content field – the value with which the output value of each selected product is replaced.

Application rules

Mergado provides a list of predefined rules, which can be used to manipulate products’ data. This is useful if you want to create a set of rules according to the eshops feed, the current day, the weather, etc. In many cases it is very useful to define your own rule which can be instantiated by an app.

Warning: If a rule applied to an input query hides a product and although the product is in some queries of other rules, then still on that product there will not be applied any other rules.

Creating application rule

Each application can define rules by exposing a URL. First you need to create such rule in the Mergado Developers center. In your App select Rules in left menu. You must provide:

There are also optional settings which you can use to specify version of product data representation, filters for product elements or URL address where settings request will be sent.

Rule settings per each application

For each rule application, it is possible to modify some of the above settings. Before the rule application settings request is sent to the settings_url specified in the Mergado Developers center, the default settings of the rule are included. Applications can than modify some of the provided settings thus, modifying rule settings for this specific rule application. If settings_url is provided and if an error occurs during the settings request, or if Mergado does not receive a response, rule application will be terminated. Fields included in settings request:

{
   "project_id": "XXXX",
   "runtime_format": "heureka.sk",
   "rule_id": 1,
   "apply_settings": {
      "app_full_name": "application_name",
      "app_rule_type": "apps.application_name.dev.rule",
      "app_url_address": "https://application_name.cz/rule/input",
      "version_for": "2023-09-07",
      "element_sending_policy": "advanced_settings",
      "element_regex": "(*)",
      "mergado_xml_equivalent": ["ITEM_ID"],
      "values_to_extract": [
         "IMG_URL { @@POSITION = 1 }",
         "IMG_URL { @@POSITION = 5 }"
      ],
      "max_payload_size": 1000000
   }
}

Explanation of the fields:

• project_id - ID of the project. Changes in products by application rule are applied to this project.
• runtime_format - Name of the format in which the products are represented when this rule is applied.
• rule_id - ID of rule that will be applied on specified project.
• apply_settings - Settings of application rule.
  • app_full_name - Name of application that rule belongs to.
  • app_rule_type - ID of rule specified in Mergado Developers center.
  • app_url_address - URL address where product data will be sent.
  • version_for - Version of product data representation. Date must be in YYYY-MM-DD format.
  • element_regex - Regex by which project elements will be filtered.
  • mergado_xml_equivalent - Mergado XML elements whose equivalent for the given format will be sent from Mergado.
  • values_to_extract - List of element paths or variables for which product data will be sent. This field provides functionality only if version >= 2022-09-10 is used. If any number of element paths are provided, the data request will include an additional field called extracted_values containing the product data found for those specific element paths.
  • max_payload_size - Maximum number of characters sent in one data request. Maximum possible value is 15000000 characters.
  • element_sending_policy - Policy of element filtration. There are only 3 possible values:
    • advanced_settings - Filter product data by using element_regex and mergado_xml_equivalent.
    • send_all_elements - Send all product data.
    • send_no_elements - Send no product data at all.

In response, application can change blue colored fields. Any other modification will be ignored and default value sent to application will be used.

Instantiating new rule via Mergado API

Send HTTP POST request on https://api.mergado.com/projects/<project-id>/rules/ URL.

with body:

{
	"name": "Rule name",
	"type": "app",
	"data": {
		"app_rule_type": "apps.appname.dev.processing",
	},
	"...Some data skipped, see Mergado API doc for more info...",
}

Explanation of the fields:

• name - Rule name as it will apear in Mergado UI for users
• type - This is type of the rule
• data.app_rule_type - This is app_rule_type you can find in Mergado Developers center

Versioning

Now application rule supports 2 versions. Version until 2022-09-10 and from 2022-09-10. You can specify version you want to use in Mergado Developers center by specifying version parameter. Every date >= 2022-09-10 means, that MERGADO will use new version and sent product data in the new representation.

Processing of application rule for version until 2022-09-10

{
    "rule_id": "12",
    "project_id": "1",
    "apply_log_id": "501",
    "request_id": "1BAC92",
    "current_format": "heureka.cz",
    "data": [
        {
            "id": "123",
            "created_at": "2016-03-27T17:18:45+00:00",
            "updated_at": "2016-04-28T17:18:50+00:00",
            "output_changed_at": null,
            "data": {
                "ITEM_ID": "890",
                "PRODUCTNAME": "Amazon Kindle Paperwhite 3",
                "DELIVERY_DATE": "0",
                "EAN": "0848719006099",
                "PRICE": "2525.10",
                "IMGURL": "https://app.mergado.com/img/890.jpg"
            },
            "metadata": {
                "..."
            }
        },
        {
            "id": "234",
            "created_at": "2016-01-07T13:56:42+00:00",
            "updated_at": "2016-04-28T17:18:50+00:00",
            "output_changed_at": null,
            "data": {
                "ITEM_ID": "530",
                "PRODUCTNAME": "Epson LX 100 EPS",
                "DELIVERY_DATE": "3",
                "EAN": "0010313601581",
                "PRICE": "185.10",
                "IMGURL": "https://app.mergado.com/img/530.jpg"
            },
            "metadata": {
                "..."
            }
        }
    ]
}

{
    "data": [
        {
            "id": "234",
            "data": {
                "DELIVERY_DATE": "0"
            },
            "metadata": {
                "..."
            }
        }
    ]
}

Processing of application rule for version from 2022-09-10

{
    "rule_id": "12",
    "project_id": "1",
    "apply_log_id": "501",
    "request_id": "1BAC92",
    "current_format": "heureka.cz",
    "data": [
        {
            "id": "123",
            "created_at": "2016-03-27T17:18:45+00:00",
            "updated_at": "2016-04-28T17:18:50+00:00",
            "output_changed_at": null,
            "data": {
            	"elements":{
			"ITEM_ID": [{"value": "111223"}],
			"PRODUCTNAME": [{"value": "Amazon Kindle Paperwhite 3"}],
			"DELIVERY_DATE": [{"value": "0"}],
			"IMGURL": [{"value": "www.image1.com"}, {"value": "www.image2.com"}],
			"GIFTS": [{
				"attributes": {"ID": {"value": "KVP-12"}},
				"elements": {"GIFT": [{"value": "Free Delivery"}]}
			}]
               },
               "attributes": {
               		"TOP_LEVEL_ATTRIBUTE": {"value": "123456"}
               }

            },
            "extracted_values": {
                "IMAGE { @@POSITION = 2 }": ["www.image2.com"],
                "GIFT { @ID = 'KVP-12' }": ["Free Delivery"]
            }
            "metadata": {
                "..."
            }
        },
        {
            "id": "234",
            "created_at": "2016-01-07T13:56:42+00:00",
            "updated_at": "2016-04-28T17:18:50+00:00",
            "output_changed_at": null,
            "data": {
            	"elements":{
		        "ITEM_ID": [{"value": "5336"}],
		        "PRODUCTNAME": [{"value": "Epson LX 100 EPS"}],
		        "DELIVERY_DATE": [{"value": "3"}],
		        "EAN": [{"value": "0010313601581"}],
		        "IMGURL": [{"value": "www.image4.com"}, {"value": "www.image5.com"}],
		        "GIFTS": [{
		        	"attributes": {"ID": {"value": "KVP-175"}},
		        	"elements": {"GIFT": [{"value": "Discount 10$"}]}
		        }]
               }
            },
            "extracted_values": {
                "IMGURL { @@POSITION = 2 }": ["www.image5.com"],
                "GIFT { @ID = 'KVP-12' }": ["Discount 10$"]
            }
            "metadata": {
                "..."
            }
        }
    ]
}

{
    "data": [
        {
            "id": "234",
            "data": {
            	"elements": {
		        "ITEM_ID": [{"value": "11111"}],
		        "IMGURL": [{"value": "www.image4.com"}, {"value": "www.image5.com"}],
		        "GIFTS": [{
		        	"attributes": {
		        		"ID": {"value": "KVP-175"}
		        		"DURATION": {"value": "3 weeks"}
		        	},
		        	"elements": {"GIFT": [{"value": "Discount 10$"}]}
		        }]
                }
            },
            "extracted_values": {
                "IMGURL { @@POSITION = 2 }": "www.new_image.com",
                "ITEM_ID": "777",
                "GIFTS { @@POSITION = 2 } | GIFT": "Discount 15$"
            }
            "metadata": {
                "..."
            }
        }
    ]
}

Retrying on Errors and Delaying Requests

Whenever your app returns 4xx or 5xx HTTP status code, the application of rules fails and the whole project’s rebuild is interrupted. Users than see this error in the UI.

Our server attempts retrying (5 times at most) when the app returns a 502, 503, 504 or 429 status code on an app’s rule application. The retry can be affected when the app responses with a 429 status code with a Retry-After header. For more information on this header, see Mozilla docs.

To delay another request (to prevent application overload) pass Mergado-Please-Slow-Down: true header. The delay will increase with every such response.