Introduction
With Infradox version 32.36 or later, you can use AI API’s to have images analysed in order to get photo keyword suggestions. The infradox platform currently supports four major AI keywording API’s:
Google Cloud Vision, Amazon AWS Rekognition, Microsoft Azure Vision, and EP Cloud.
You can configure one or more of the above API’s, and you can then select the active API in back office. Currently only English is supported, additional languages may be added later.
AI keywording API config (keys shown are for illustration purposes only, ie not real keys)
Demo videos / Screen recordings
We have made screen recordings to show how the AI keywording function works in the HTTP uploader, the metadata edit dialog and in the batch editing dialog. If you want to view these recordings then create a support ticket on xpertise.zendesk.com or send an e-mail to support.
Recommended reading
- Working with the controlled vocabulary
- Data processing rules
- The metadata repository
- Adding keywords from similar files
Initial API configuration
Note that if you want to try whether or not an API suits your needs / is best for your kind of content, you may not have to create an account first. There are test pages that let you upload your own images to see how it works – below are URL’s to test pages on Google and Microsoft Azure:
- Google’s Cloud Vision API “image annotations”
- AWS Rekognition “detect labels”
- Microsoft Azure Vision API “image tagging”
If you want to see which keywords are generated with the EP Cloud API, then send a few 1000 pixel images to support – a test page is not publically accessible at this point. EP Cloud does seem to return good results for general lifestyle stock photography.
Once you have created an account for each of the API’s that you want to use or test, you’ll need to send the API details to Infradox support. And Infradox staff will then add the configuration data for you on your web server. Once this has been done, you can go to “API Integration” in backoffice and you can select and configure the active API that you’re going to be using.
You can download the set up instructions (PDF) for the different API’s below:
The EP Cloud API is configured for you by Infradox support, you can’t do this yourself. If you want to use/test this API then create a support ticket for it.
Enabling AI keyword suggestions in the uploader, the metadata edit dialog and the batch dialog
HTTP Uploader
To enable keyword suggestions in the HTTP uploader, go to Site configuration, HTTP Upload settings and select either
the “Enabled – display as suggestions” (A) or the “Enabled – merge with extracted keywords” (B) radio button next to the AI API keywords label.
- Keywords are displayed underneath the keywords input field. Click either “Add all” or click on single keywords to words to the input field. Keyword suggestions can also be removed from the list first, e.g. to remove the words that you don’t want before clicking “Add all”.
- Keywords are automatically added to the keywords field, or merged with existing keywords if keywords are extracted from XMP/IPTC. Note that every image that is uploaded with the uploader will always be analysed for automatic keywording (i.e. every upload automatically results in an API request). Until the maximum number of requests has been reached.
Metadata edit dialog
To enable fetching keyword suggestions in the metadata edit dialog, find the keywords field in the Metadata repository (backoffice) and click on it to open the properties dialog. On the Vocabulary tabsheet, tick the radio button “Enable in metadata dialog” for the “AI keyword API” setting. Or if you want to enable batch AI keywording as well, then tick the radio button “Enable in metadata and batch dialog”.
With this setting enabled, a link is displayed underneath the keywords field in the metadata edit dialog. Click the link to fetch keyword suggestions from your AI API. The results are cached on the server and displayed as clickable keywords (including the option to remove words and to “Add all” words).
The next time the link is clicked, the results will be read from the cached file on the server. To force submitting the file to your AI API again (instead of getting cached results), hold down ctrl (Windows) or cmd (Mac) when you click on the link.
Batch AI keywording
To enable this function, find the keywords field in the Metadata repository (backoffice) and select the “Enable in metadata and batch dialog” radio button for the “AI keyword API” setting.
The “AI keywording” button will now show in the batch edit dialog. Click that button to edit multiple files at the same time.
AI keywording batch dialog
Costs
The API integration page will show you for each API how many requests you can execute without any costs, and what the costs are if you exceed the number of free requests. To prevent or to limit costs (e.g. while you’re still testing or deciding which API best suits your needs), you can enter a number in the “Maximum requests” input box. Once this number has been reached, further requests will not be executed until the next time period (i.e. day or month).
All API requests are logged and you can check how many requests have been made via Logs, Statistics in backoffice. Depending on the API that you are using, you’ll have to add credits to your account beforehand or you’ll be billed at the end of the month by the API provider (i.e. Google, Amazon or Microsoft). In case of the EP cloud API, the costs are added to your Infradox hosting invoice.
Other configuration options
Go to Site configuration > API integration in backoffice and open the section “AI keywording”.
- Besides the maximum number of requests per day or month, you can also configure the maximum number of keywords that you want to show as suggsted keywords.
- If the API of your choice returns generic keywords that you never want to display as keyword suggestions, then enter those in the “Word exclusions” box. For example “photography, image, picture, body part, head, foot,feet,hand,hands”.
- If you feel that you get many irrelevant suggestions, then you can use the “Minimum score (confidence)” setting to tweak your results. Based on our own tests, we think that it’s best to – at least initially – leave this setting at “Don’t care”, with a maximum of 50 keywords.