The API engine is a module used for custom data integration with any data provider our clients work with.
A custom data integration (aka API integration) is a connection between SpotMe and a third party provider to sync data (for example, participants, sessions, abstract, etc.) with a simple click. The sync can get data from the provider and import it into SpotMe or can push data from SpotMe to the provider (aka 2-way sync, for example, used for sessions registration).
On Backstage, go to the API module, navigate to API credentials and click on “[API] Credentials”. Then click and edit the value for the fields presented (for example: account, username, password and event code). The endpoint is pre-configured. To test the credentials, go to the next step.
The mapping consists of "bridges" that you build between the API and Backstage, so that the data from the API is imported in the relevant Backstage fields (ex. the participants' first names in the API are imported in the "fname" field in Backstage).
Each API comes with pre-set mapping. You can either manually edit the mapping, or import it from a previous event using the same API.
Import mapping from a previous event
- Go to "API Data" -> "Data Configs" on the previous event and click "Export XLS".
- Remove the data configs for which you do not need to import the mapping and save the file
- Import the XLS file in the new event in "API Data" -> "Data Configs".
Manually edit the mapping
Click on the data type. Under "API Data Mappings" you will see all the fields already mapped.
- Edit a field mapping: click on a field and edit the values (you need to specify at least the API field and the corresponding SpotMe field it should be mapped to).
- Create a new field mapping: click on "+" under "API Data Mappings" (you need to specify at least the API field and the corresponding SpotMe field it should be mapped to). Don't forget to save your changes!
- Remove a field mapping: click on the red X on the right of the field.
Once the mapping is edited as desired, make sure to preview the corresponding data type(s) and check that there is content imported for the fields in the "Synced Data" tab (see details below in the Preview section).
For each API field that you map to a SpotMe field in the Data Configs, you need to have a corresponding metadata in Backstage (otherwise the data cannot be displayed in Backstage).
If you map API fields to SpotMe fields that do not exist yet, make sure to create the corresponding metadata (person metadata, session metadata, presenter metadata etc.) in Setup -> Manage metadata. The name of the SpotMe field in the API mapping must correspond exactly to the name of the metadata.
Preview allows to check the data before actually importing it, to make sure that the expected data will be added to the event.
Go to Data Config: all pre-configured Data Configuration types are available here (Participants, Sessions, Sessions, etc.)
Click on “Preview” on one of the Data Config that will be used to download the preview file. Make sure to download the report and check the data:
- The "Stats" tab gives you an overview of the changes
- The "Raw API Data" tab shows all the data that is included in the client's system (with all fields, including the ones that are not mapped)
- The "Synced Data" shows the data as it would be imported
- Duplicated Ids (failed) shows a list of records that are duplicated in the endpoint and that would be then skipped in the sync
If you don’t see the 4 tabs, or see an error message, it means that something is wrong with the credentials you have received from the data provider.
Please note that auto-sync should not be set up unless necessary, since it causes additional stress to our servers and the maximum frequency is every 15 minutes.
- If you are using the API engine package v 1.1.0 or newer: you can enable an auto-sync yourself on any of the data configs in the API.
- Click on a data config, and in "Auto-sync frequency" select the frequency you need. If nothing is selected in the dropdown list, the auto-sync will be enabled with a 60 minutes frequency by default.
- On the data configs page, click on "Auto-sync" for the desired data configs. You will get a confirmation popup, and the enabled Auto-sync will be highlighted in green.
- The auto-sync will automatically stop 3 days after the end date of the event. If you need it to run longer, please contact Backstage Support.
If you are using the API engine package v 1.0.9 or older: to have an auto-sync set up, please contact Backstage Support and mention which data configs need the auto-sync, to which frequency, and the start and end dates.
Automated sync reports
If you are using an auto-sync, you may want to receive automated sync reports via email, especially to be warned about any sync error. To set up an automated sync report:
- Click on a data config
- On the right side, "Email Subscribers", click "Create"
- Enter the name of the subscribers list, the event for which you want to get an automated report (Any sync / Error occurred during sync / Never), and add the email address(es) of subscribers
- Make sure to add the subscribers list on each data config for which you want to get an automated report (reports are generated separately for each data config)
- You can also create several subscribers lists, and attach them to different data configs
Please note this functionality is available on data-api 1.1.1 and higher.
Warning! If you need to update the API package(s), please note that the configurations (aka mappings) and credentials will be overwritten.
To avoid overwriting data, you export the configurations and mappings as XLS, update the package and then import the XLS.
Remove outdated data
The way outdated data is handled depends on which API is used. For some APIs, outdated data is handled with a status that is passed on to Backstage (ex. cancelled participants are imported with status "cancelled" in Backstage). For others, the "Remove outdated data" parameter is ticked on the data configuration in the API, and outdated data will be deleted upon next sync (ex. cancelled participants are deleted from Backstage).
Exclude data from the sync
If you need only data matching certain criteria to be pulled into Backstage, you can use the filters available in each data configuration:
For example, you can choose to pull only the participants' records for which "Company" equals "SpotMe"; or the sessions records for which "Title" does not equal "Break". Only the data matching the filters will be imported. The data excluded by the filters will still appear in a separate tab of the data preview file, but will not be pulled.
Prevent manual changes from being overwritten by sync
If you make any changes (either manually or with an XLS import) to data that comes via API, syncing again the data via API will overwrite your changes. There are 3 ways of preventing data from being synced via API:
- Un-map a field to exclude this one field from syncing for all items of a certain data type (ex. un-map the 'job title' field to prevent the API to sync job titles for all participants)
- ‘fp_protect’ is to exclude one field from sync for one particular item (ex. prevent the API to sync the job title of one specific participant)
- ‘exclude_from_sync = true’ is to exclude all fields from sync for one particular item (ex. prevent the API to sync all profile fields for one specific participant)
Please note: a "field" is a metadata in Backstage (person metadata; session metadata; presenter metadata etc.); an "item" is a record for a certain data type (a participant profile; a session; a presenter etc.)
Un-map a field
Un-mapping a previously mapped field allows to protect this field from being overwritten, by excluding it from the next sync via API.
You can un-map any field yourself: in API Data, click on the data type (eg. participant), scroll down and remove the corresponding field mapping. Make sure to save!
Whenever you want to manually change one field for a single item, you need to "fp_protect" your change from being overwritten by future API sync.
- Create a new metadata for the data type you need as follows:
- Go to the item(s) you want to manage manually
- Click on + next to Protected field(s)
- Type the key of the metadata you have manually edited (in this case, fp_status)
Now you can safely sync and your manual change won't be overwritten.
If you need to prevent the API from syncing all fields of a certain item (ex. all fields for a particular session), you can create a metadata called "exclude_from_sync" in the corresponding data type (ex. Setup -> Manage metadata -> Session).
If this metadata is set to TRUE for an item (ex. exclude_from_sync=TRUE for a particular session), then all fields for this item (ex. title, day, location etc.) will not be overwritten by the next API sync.
Multiple credentials set in the same event
It may be necessary to sync data from different sources, for example an event has multiple locations and the provider has created different endpoints for each one. This can be handled by creating multiple credentials and mapping configs.
What do I do if my Credentials are not working?
Main reasons why Credentials are not working: the provider gave you the incorrect Credentials; our API’s IP is not whitelisted
What do I do if specific data is missing?
First, make sure that the data was synced.
Then, report the problem to Backstage Support indicating as much information as possible, including examples of missing data.
What do I do if the preview indicates that there is no data?
First, check that the credentials have been saved and they are correct. Then confirm that there is already data in the endpoint(s) and that you have access to it. If you can't still get data, please contact Backstage Support.
I confirmed that there are photos, why do I get 0 in the preview?
You need to have synced participants/presenters before previewing or syncing photos. The photos sync requires the corresponding data to be available in Backstage. If you can't still get photos, please contact Backstage Support.
I confirmed that there are sessions registrations, why do I get 0 in the preview?
You need to have synced participants and sessions before previewing or syncing sessions registrations. The sessions registrations sync requires the corresponding data to be available in Backstage. If you can't still get sessions registrations, please contact Backstage Support.
What do I do if there are some participants in the API that do not need to be imported in Backstage?
You can ask the provider to set the participant's ID (field mapped to fp_ext_id in Backstage) as blank ; or to use the same participant's ID for all participants who should not be imported. The API will not import participants records with blank or duplicated fp_ext_id (those records will show in a separate "Duplicated Ids (failed)" tab of the data preview).