ERP & Logistics Suite

Black Box

The black-box is a tool that will help you configure your mapping seamlessly. It is composed of 3 major features.

Input Data

Library

Output Data

Settings

In this section, you will be able to define the source of the data.

Profile settings

• Enabled
  You can enable/disable the profile.
  Note that the profile is disabled by default when it results from a duplication.
  
• Profile Name
  Only alphabetic and number are allowed + hyphens (-) and underscores (_).
  The profile's name must be unique.
• SQL mode
  When SQL mode is enabled, no data are updated. Running the profile will only produce a SQL file. This file could be executed directly in your database manager.
  If you activate SQL mode, you'll need to fill in several fields:
• SQL file name
  Name of the SQL file to generate.
  

  XML_sample.sql

• SQL file path
  Is the path where the SQL file will be generated (relative to Magento root folder). 
  

  var/sample

• Profile method
  You can choose the action that the profile will trigger:
  Update products only
  Import new products only
  Update products and import new products
• Filter lines
  Leave empty to export all lines:
  

  e.g:

  means all lines will be imported

  
  Type the numbers of the lines you want to import, making sure to separate each line or range with a comma (,):
  

  e.g: 2,4,6,8,10

  means lines 2,4,6,8,10 will be imported

  
  Use a dash (-) to denote a range of lines:
  

  e.g: 8-10

  means lines 8,9,10 will be imported

  
  Use a plus (+) to import all lines from a line number:
  

  e.g: 4+

  means all lines from line 4 will be imported

    
  Use regular expressions surrounded by # to indicate a particular group of identifiers to import:
  

  e.g: #ABC-[0-9]+#
  means all lines with an identifier matching the regular expression will be imported 

When the Profile method option is on Update products and import new products, in the Mapping & Rules tab, you can define for each Target attribute dropdown if the value applies to the new products, or/and to the existing products:

• New products only
• Existing products only
• Both, new and existing products

File Location

• Magento File System
  The file is stored in the Magento root directory or a subdirectory of the Magento root directory.
  File path relative to the Magento root folder.
  See how to use regular expression in the file name.
• FTP server
  The file is stored on a remote server available from FTP or SFTP. Do not prefix the domain name or the IP with FTP://.
  You'll be able to check if the FTP connexion is valid.
  The file path is relative to the FTP user root folder.
  You can also use regular expression in the file name when it is stored remotely via FTP.
• URL
  The file is stored on a remote server available from HTTP or HTTPS.
  

  The file path is the URL of the file.
  

  http://www.website.com/Inventory/AvailableInventory

    
  If you' are using HTTP authentication, you must add the login and the password before the URL in the below format:

  http://login:password@www.website.com/Inventory/AvailableInventory

  If you are using a specific port, you can use the below format:
  

  http://login:password@www.website.com:port/Inventory/AvailableInventory
  

• Webservice
  The file is available on a remote server accessible as web services.
  The extension uses the CURL PHP library to connect the web service.
  The file path is the URL of the web services.
  You can provide the web service a login/password couple and any parameters formatted as URL parameters, as follows:
  

  parameter1=value1&parameter2=value2&parameter3=value3

• Dropbox
  You can generate your token from your Dropbox account https://www.dropbox.com/developers/apps
  The file path is the URL of the dropbox service.
  

File Type

Allows you to specify the file type used for the update, XML, JSON, or CSV.

• File type
  You have the choice between CSV, XML, and JSON files.

For CSV files, define:

• Column separator
  Define the column separator.
• Text delimiter
  Select the text delimiter.
• The first line is a header
  Specify when the first line is a header.
• Magento export file
  Magento default export files are made of empty rows with values related to main rows, activating this option will merge the values into the main rows.

For XML and JSON files, define: 

• Path to products
  Fill in the path that targets the products. The path is case-sensitive.
  See how to define the path to your products for XML or JSON files. 
• Data structure
  You have two options: Automatic detection or Predefined structure
  The automatic detection fits for simple files made of only one nesting level.
  If you've selected the Predefined structure, see how to define your own file structure for XML or JSON source file. 

Post Process Action

• Action
  Do Nothing
  Delete the import file
  Move import file
• Move to folder
  Define the file path where you want to move the import file, relative to your Magento root folder.
  

• Run indexers
  You can choose to run the indexers after the import or not:
  No: no indexer will run and you will have to reindex manually from the CLI
  Only the required indexers: automatically decides which indexers must run
  Only the selected indexers: let you decide which indexers to run
  
• Indexer to run
  Choose the indexers to run: 
  

  catalogrule_rule
  catalogrule_product
  catalog_product_price
  cataloginventory_stock
  inventory
  catalog_product_attribute
  catalog_url
  catalogsearch_fulltext
  catalog_category_product
  catalog_product_category

Advanced Settings

System Settings

• Do Nothing
• Disable the product
• Delete permanently the product
• Mark the product as out of stock
• Mark the product as out of stock and set qty to 0

When you decide to mark products that are missing from the data file as out of stock, you must select the targeted sources impacted by the action.

Great Lake Store

When a product is imported using Mass Product Import & Update, it is associated with the profile. If you've selected action for missing products, you'll need to choose the targeted products: 

• Only products related to current profile: products that are imported/updated with the same profile.
• Only products not related to current profile: all the products that are not created/updated by the profile.
• All products

Stock Settings

From the Stock Settings, you can define:

• Automatic stock status update
  You can choose to automatically update the stock status (in stock / out of stock).
• Use relative quantity update
  Quantity will be added/subtracted to/from the current quantity.
  

Image Settings

• Images location
  Http server (URL)
  Magento File System
  Ftp server
  

Depending on what you've selected as the location of the images, you'll need to fill in several fields.

For the FTP server:

• Use SFTP
• Use active mode
• Host
• Port
• Login
• Password

For the FTP Server and the Magento file System:

• Path to images directory

Category Settings

• Create categories on the fly
  Yes or No.
• New categories active by default
  Yes or No.
• New categories included in menu by default
  Yes or No.
• Category tree auto-detection
  To automatically detect the category tree.
  You need to use the complete path to the category. The category levels must be separated with slashes: A/B/C.
  For example, instead of "Shirts", you should write:
  

  Default Category/Men/Shirts

• New categories are children of
  Choose the parent category.

Configurable Product Settings

Create parent of configurable products on the fly
You can also create/update a configurable product from the simple products available in the source file by enabling this option.

When that option is enabled, in the Mapping & Rules tab, you can find for each Target attribute dropdown an option for the configurable products to create on the fly. You have the choice to apply the settings on: 

• The current product only
• The configurable product created on the fly only
• Both, configurable product created on the fly and simple product associated

System attributes of the configurable products created on the fly must receive a value either by mapping a field from the source file for the simple product and the configurable product or for the configurable product only.

So that the configurable product and the simples variants are linked, the following attributes must be mapped:

• the SKU of the parent product or the SKU of the children products
• the configurable attributes

Mapping & Rules

In this section, you can map each column of the file with one of the stock attributes.

Column Mapping

Each column can be mapped with one of the stock attributes. Note that at any time, you can preview the input and output data in the black box directly.

• Target Attribute 
  Select the target attribute from the ones available in your system.
  You can find all the attributes grouped by type in our documentation.
  The first line is where you define the identifier of your products. The identifier must be unique, 
  

  SKU

• Data source
  In the Data source, you can define the attribute source that matches with the target attribute from your source file. 
  

  id

• Default value
  Sometimes, some columns can be missing. For example, when updating all stocks, the file does not necessarily include all stock attributes (backorders, minimum quantity in the cart...). In that case, you can choose custom value in the Data source and apply a default value. 
  

  yes

QUICK OVERVIEW OF THE INTERFACE

1. Add as many columns as you want by clicking on + ADD A NEW ROW. In many cases, the update file may contain more columns than necessary; you don't need to add these columns and match them.
2. DELETE a row easily.
3. Change the order of each row using the drag and drop.
4. Deactivate a line while conserving the settings by clicking on the LINK symbol.
5. Add your own custom script </>. These scripts can be previewed in the output. Note that when there is a script filled in, </>  is underlined. See how to use the PHP API. 
6. Add your own replacement rule by clicking on the  b icon. These rules can be previewed in the output. Note that when there is a rule selected, a b is underlined. See how to create your replacement rules. 
7. Apply the color of your choice to the row.
8. Apply a label to the row.
9. Define to which store view(s) you want to apply these settings by clicking APPLY TO DEFAULT VALUE.
10. Define to which product(s) you want to apply these settings by clicking APPLY TO THE CURRENT PRODUCT ONLY.
    For this setting to appear, the Create parent of configurable products on the fly option must be activated.
11. Define to which product(s) you want to apply these settings by clicking Apply to Both, new and existing products.
    For this setting to appear, the Profile method must be set on Update products and import new products.
12. Preview your file by clicking INPUT DATA.
13. Find all the attributes grouped by type in the LIBRARY.
14. Check the data as they'll be updated by clicking OUTPUT DATA.

Scheduled tasks

You can schedule the profile to be executed for all days of the week and with a default interval of 30 minutes between each execution.

You have the possibility to change the cron task interval from the General Settings tab. 

Action History

In the Action History tab, you can find a history of your previous generation and configurations. 

You'll find: 

• ID
  It is the ID of the history line. 
  
• Version ID
  It is the ID of the profile configuration version.
  
• Origin
  The origin of the action can be: Backend / Cron / CLI / API
• Username
  It is the name of the user behind the action.
• Action
  The available actions are: create / generate / update / delete
  
• Details
  If you click on More details, you can see the detail of the profile update from a grid.
  

  

  If you click on a line of the above grid, you can check the differences of the updated field: red parts are the removed data and green parts are the inserted data.
  

  

  
  
• Date
  It is the date of the modification.

Read more documentation about the Mass Product Import & Update module.

The Tool Box or so-called Black Box

Mass Order Export provides you with a tool to help you configure your profiles as you want. When editing a profile you'll find a toolbox on the right of the window.

The toolbox is inevitably a very useful tool. This will help you to configure your profiles more quickly.

Here are the different options of the black box.

Preview

Library

Profile Configuration

When creating or editing an export profile in Mass Order Export, you have to be aware of some basic settings.

Export file(s) Configuration

You can start with the file configuration of the file.

• File name:
  It is the base name of the exported files. Note that there is no need to add any file extension (XML, CSV, TXT) as the extension does it automatically.
  

  file

• Encoding type
  Choose the encoding type you want to use between:
       UTF-8: It is the most common format
       UTF-8 without BOM: See more about BOM
       ANSI
• File name format
  Your file name can be completed with the current date at the time of the exportation. Different formats are available.
  

  file.txt

  2019-07-03-file.txt
  file-2019-07-03.txt

• Create one file for each order
  Choose to export one order per file or not.
• File name suffix
  When choosing to export one order per file, you need to select a file name suffix between the order increment ID, the Magento internal order ID or the module internal auto-increment.

Export options

In that section, define how you want the file to be exported.

• Start with increment ID
  You can add an ID from which the orders should be exported.
  

  000000002

  To export all the orders, you can leave this field empty.
• Register the latest exported increment ID
  If enabled, this will automatically be updated with the latest ID.
• Mark each order as exported
  If enabled, this will mark each exported order with the name of the profile. Note that an order can receive several flags.
  

  Exported to

  ERP/CRM export/import

• Export only unmarked order
  If enabled, only orders that are not yet marked will be exported. Note that the orders must also match with the filters.
  

• Update the order status
  You have the possibility to update the order status.
• New order status
  Choose the status to apply to the exported orders. 
  For example:
  

  Processing

• Message in the order history
  You also have the possibility to add a comment to the history.
  

  Order exported

  If you don't want to add any message, you can leave this field empty.

Filters

In the Filters tab, you can define the types of orders and customers you want to include in your export profile. You have also the possibility to create more advanced filters that will best meet your needs.

Store view

In this tab, choose the store view from which to export the orders. At least one store view must be selected

Order status

This section allows you to specify the type of order you want to export. At least one status must be selected.

Customer Group

In this tab, choose the customer group of the orders you want to export. At least one customer group must be selected.

Advanced Filters

In that tab, you can create more precise filters. For example, you can choose to export only one specific order by setting a filter:

order > increment_id | = | ID of your order

You can also export orders placed by customers whose last name ends with 'son':

order > customer_lastname | like | %son

You have a choice between several options:

• =, >, <...
  Equal to, greater than, lower than, greater than or equal to, less than or equal to, different from.
• LIKE / NOT LIKE
  like: starts/ends with the character string / includes the character string.
  Not like: doesn't start / end with the character string / doesn't include the character string.
• IS NULL / IS NOT NULL
  is null: the value is null.
  is not null: the value is not null.
• IN / NOT IN
  in: is in (selection of several options).
  not in: is not in (selection of several options).

• REGULAR EXPRESSION / FIND IN SET
  Use regular expressions to describe a sequence of characters. 

Template

In this tab, you'll be able to configure your profile template according to your needs. Mass Order Export allows you to create different kinds of file:

• XML
• CSV
• TXT
• TSV
• DIN

XML Export file settings

You can select the XML file type.

• File type
  XML
• Enclose XML tag content inside CDATA
  When enabled, this encloses all node values with CDATA and avoids issues with special characters such as < (lower than), > (higher than) or & (ampersand). Note that this is recommended.

XML file template

An XML template is made up of 3 parts:

• Header
  Is at the start of the file:
  

  <orders>

• Body 
  Is iterated for each order included in the file.
  

  <order>
  ...
  </order>​​

• Footer
  Is at the bottom of the file:
  

  </orders>

The XML structure allows you to use iterative syntax in different parts of the template, as follows:

<items>
<?php foreach($products as $product): ?>
<item id={{product.item_id}}>{{product.name}}</item>
<weight>{{product.weight}}</weight>
<?php endforeach; ?>
</items>

In most cases, orders contain more than one product, so it may be useful to create an iteration that loops for each ordered product. All the content between the opening and closing markers will be repeated for each product and all dynamic values will be replaced:

<?php foreach($products as $product): ?>
...
<?php endforeach; ?>

These iteration markers can be used for all order instances:

<?php foreach($payments as $payment): ?>
...
<?php endforeach; ?>
<?php foreach($invoices as $invoice): ?>
...
<?php endforeach; ?>
<?php foreach($shipments as $shipment): ?>
...
<?php endforeach; ?>
<?php foreach($creditmemos as $creditmemo): ?>
...
<?php endforeach; ?>

CSV/TXT/TSV/DIN Export file settings

The file type can be CSV, TXT, TSV, or DIN. You have the possibility to create basic or advanced files.

• File format: Advanced format
  It consists of a complex file structure divided into several blocks.
  

  

• File format: Basic format 
  It consists of one optional header and several rows applying the same pattern.
• Include header
  Allows you to include or not the column's name in the first row of your file.

• Delimiter character
  Lets you choose which kind of delimiter you want to use to separate your fields.
• Enclosure character
  Character that surrounds each field of the rows.
• Escape character
  Lets you choose which kind of enclosure character you want to use for your data.

CSV/TXT/TSV/DIN file template

The template tab will be updated to match the file type you've selected.

Basic file format

Basic format for txt-like files consists of one optional header and several rows applying the same pattern.

• Extra Header
  These are additional row(s) of headers that will be added at the very top of the file.
• Header
  Is for the attribute name that can be displayed at the top of the file.
• Field Pattern
  Works in the same way as in XML files. Like for XML files, you're allowed to use PHP code if you want to customize the pattern. You just have to write your PHP code in one single line.
• Insert a new field
  You can add a field by using this button. The fields can then be re-ordered easily by using the drag and drop arrows.
• Extra Footer
  These are additional row(s) that will be added at the very bottom of the file.

The main difference with the XML mode is that you can't use any iteration instructions. Instead, Mass Order Export uses an auto-consolidation internal process. This simply means if you try to get the value of any attribute of an instance (product for example) that contains several items, then the extension will automatically duplicate the TXT/CSV line for each item.

With the following configuration:

Logically the module should export one line for each order, but some orders may contain several product items. The extension will create several lines for the same order, as in the following output example:

Order#         Product Name          Product Sku
#10000001      my product A          AAA
#10000001      my product B          BBB
#10000001      my product C          CCC
#10000002      my product D          DDD
#10000003      my product B          BBB
#10000003      my product E          EEE
...

Advanced file format

Advanced format for txt-like files consists of a complex file structure divided into several blocks.

• Extra Header
  These are additional row(s) of headers that will be added at the very top of the file.
• Body
  Is iterated for each order included in the file.
• Extra Footer
  These are additional row(s) that will be added at the very bottom of the file.

This format allows you to use iterative syntax in different parts of the template, as follows:

<?php foreach($products as $product): ?>
PRODUCT=
{{product.sku}}
{{product.qty_ordered}}
{{product.base_row_total output="number_format($self,2)"}}$
<?php endforeach; ?>

Output

With Mass Order Export, you have many possibilities to export your profiles.

Storage settings

The first solution is to store your file directly on your server.

FTP settings

You can also configure the FTP settings in order to upload the generated profile to an external FTP server. For example:

• Upload by FTP
  Select Yes if you want to upload your profile using FTP.
• Use SFTP
  Select Yes if you are using SFTP.
• Use passive mode
  If you are using SFTP, you don't have to care about the passive/active mode.
• Host
  In that field, fill in the host.
  

  wyomind.com

• Port
  It is the port to connect to server.
• Login
  It is your login to connect to server.
• Password
  It is your password to connect to server.
• Destination directory
  It is a relative path from the entry directory when you connect to the ftp.
  

  /var/export/ftp

Email settings

Finally, you can choose to send your profiles by email. Don't forget to separate the email addresses with a comma.

• Send by email
  You have the possibility to send the profile by email. 
• Email sender
  This is the email of the sender. 
• Email recipients
  This is the email(s) of the recipient(s). 
  
• Email subject
  Define the subject of the email. 
• Email body
  Add the content of your choice in the email body. 

You have the possibility to check the email by clicking on Test Email. 

A popup will confirm that the email has been. 

• Send all files in the same email
  You have the possibility to send all the files in the email. 
• Send all files in a zipped file
  You can add the files in a zip file.

Cron Schedule

In that tab, you can choose the days and hours when you want your profiles to be generated.

Read more documentation about the Mass Order Export module.

The Tool Box

The toolbox is very useful as this will help you to configure your update file more quickly.

Below are the different options for the toolbox.

Input Data

Output Data

You can have different notifications among which:

• Order not found
  The order cannot be found among your Magento instance.
• Order is not...
  The order cannot be shipped, invoiced...
• Conditions not fulfilled for rule ***
  The order doesn't meet the conditions for the rule.
• Trying to ...
  The order meets the conditions for the rule, and the action will be performed on the order.

Settings

Import Profile Settings

Define the settings for your profile.  

• Profile Name
  The profile's name must be unique.
• Filter lines
  Leave empty to import all lines:
  

  means all lines will be imported

  
  Type the numbers of the lines you want to import, making sure to separate each line or range with a comma (,):
  

  2,4,6,8,10

  means lines 2,4,6,8,10 will be imported

  
  Use a dash (-) to denote a range of lines:
  

  8-10

  means lines 8,9,10 will be imported

  
  Use a plus (+) to import all lines from a line number:
  

  4+

  means all lines from line 4 will be imported
  

  
  Use regular expressions surrounded by # to indicate a particular group of identifiers to import:

  #0000002[0-9]+#
  

  means all lines with an identifier matching the regular expression will be imported

File Location

This section allows you to specify where the source data file is stored. You have several options:

• Magento File System
  The file is stored in the Magento root directory or a subdirectory of the Magento root directory.
  The File path is relative to Magento root folder.
• FTP server
  The file is stored on a remote server available from FTP or SFTP. Do not prefix the domain name or the IP with FTP://.
  You'll be able to check if the FTP connexion is valid.
  The File path is relative to FTP user root folder.
• URL
  The file is stored on a remote server available from HTTP or HTTPS.
  Fill in the File path with the URL of the file.
• Webservice
  The file is available on a remote server accessible as web service.
  Fill in the File path with the URL of the web service.
  The extension uses the CURL PHP library to connect the web service. You can provide the web service a login/password couple and any parameters formatted as url parameters, as follows:
  

  parameter1=value1&parameter2=value2&parameter3=value3

• Dropbox
  Fill in the File Path with the URL of the dropbox service.
  Generate your token from your Dropbox account https://www.dropbox.com/developers/apps

• File path
  The path to the file including the file name and the extension as it appears in the file system of the server. 
  You can also use regular expressions in the file paths. 

Order Identification

Choose how your orders will be identified in the imported file:

• Order identification
  You have the possibility to choose the order id or the order increment.
  For example:
  

  3420

  or
  

  1000003421

• Identifier offset
  You need to select the field which contains the order identifier.

File Type

This allows you to specify the file type used for the update between XML and CSV.

For CSV files, define:

• Column separator
  Define the column separator.
• Text delimiter
  Select the text delimiter.
• The first line is a header
  Specify when the first line is a header.

For XML files, define:

• Xpath to orders
  In case of an update from an XML file, fill in the XPath that targets the orders. The Xpath is case sensitive.
• XML structure
  You have two options; the Automatic detection or the Predefined structure.
  The automatic detection fits for simple files made of only one nesting level such as the above file sample.
• Predefined structure
  If you've selected the Predefined structure, you have the possibility to define your own file structure. When the structure of your file is quite complex, the info may be dispatched in deep nodes and sometimes in tag attributes such as:
  

  url="something"
  

  
  
  Below is an example of complex file where you need a predefined structure:
  

  <orders>
   <order>
    <CustomerOrderId>000000028</CustomerOrderId>
    <order-shipment>
     <OrderTrackingNumber>334678MCLK</OrderTrackingNumber>
     <OrderCarrier>ups</OrderCarrier>
     <OrderTrackingURL>https://www.fedex.com/apps/fedextrack/?action=track&amp;trackingnumber=334678MCLK&amp;cntry_code=us&amp;locale=en_US</OrderTrackingURL>
    </order-shipment>
   </order>
  </orders>

  The XPath that targets the orders in the above file is /orders/order

  The JSON syntax you need to use is:
  

  {
  "incrementid":"CustomerOrderId",
  "tracking_number":"order-shipment/OrderTrackingNumber",
  "carrier":"order-shipment/OrderCarrier"
  }

Post Process Action

You have the possibility to define an action after the file is processed.

• Action
  Do Nothing
  Delete the import file
  Move import file
• Move to folder
  Define the file path where you want to move the import file, relative to your Magento root folder.
  

Rules

In that section, you can define your own rules and give them a Rule name. 

A rule is a combination from 0 to n conditions and from 1 to n actions. Rules are tested for each order corresponding to an element of the file. If the conditions are met for the current element, the actions of the rule will be executed for the order.

If there is no condition for a rule, the actions are always executed for the order.

Overview of the interface

Find below a quick overview of the Rules section. 

1. Enable or disable a rule by clicking on the LINK symbol.
2. Apply a label to the rule.
3. Add a condition to your rule by clicking on +. 
4. Delete a condition from the rule.
5. Add a new action. 
6. Delete an action from the rule.
7. Change the order of each row using the drag and drop.
8. Apply a label to the action.
9. Apply the color of your choice to the action.
10. Apply a custom script </> to the action options.
    These scripts can be previewed in the output. 
    See how to use the PHP API.
11. Preview your file by clicking INPUT DATA.
12. Check the actions that will be performed on your orders by clicking Execution preview.

Conditions

You can define conditions based on:

• the file: the conditions are based on the data available in the source file
• the order: the conditions are based on the orders data

You have the possibility to define conditions according to several options such as:

• EQUAL / NOT EQUAL
• GREATER / LOWER
• GREATER OR EQUAL / LOWER OR EQUAL
• LIKE / NOT LIKE
• IS NULL / IS NOT NULL
• IN / NOT IN

Actions

You can select the below actions: 

• Set status
  You will have to select the status you want to apply to the orders. 
  

  

• Add comment
  You can define the comment you want to add to the orders as well as if you want to send the comment email and make the comment visible on the front-office, in the customer account.
  

  

• Invoice
  You can define if you want to send the invoice and the email.
  
  

  

• Creditmemo
  You can define if you want to send the credit memo and the email.
  
  

  

• Ship
  Define the carrier to apply, the tracking number, the source to use, and if you want to send a shipping email.
  

  

• Cancel
  Define if you want to cancel the orders. 
  

  

Scheduled Tasks

You can schedule the profile to be executed for all days of the week and with a default interval of 30 minutes between each execution.

The cron interval can be configured in the General Settings from: 
Stores › Settings › Configuration › Wyomind › Mass Order Update

Read more documentation about the Mass Order Update module.