This SDK is deprecated. Please use https://github.com/visenze/visearch-sdk-swift instead.
- Overview
- Setup
- Initialization
- Solution APIs
- 4.1 Visually Similar Recommendations
- 4.2 Search by Image
- 4.2.1 Selection Box
- 4.2.2 Resizing Settings
- 4.3 Search by Color
- 4.4 Multiple Products Search
- Search Results
- Advanced Search Parameters
- Event Tracking
ViSearch is an API that provides accurate, reliable and scalable image search. ViSearch API provides two services (Data API and Search API) to let the developers prepare image database and perform image searches efficiently. ViSearch API can be easily integrated into your web and mobile applications. For more details, see ViSearch API Documentation.
The ViSearch iOS SDK is an open source software to provide easy integration of ViSearch Search API with your iOS applications. It provides four search methods based on the ViSearch Solution APIs - Find Similar, You May Also Like, Search By Image and Search By Color. For source code and references, please visit the Github Repository.
Current stable version: 1.5.1
Supported iOS version: iOS 7.x and higher
The source code of a demo application is provided together with the SDK (demo). You can simply open ViSearchExample project in XCode and run the demo.
You should change the access key and secret key to your own key pair before running.
@implementation HomeViewController {
NSMutableArray *rectangles;
}
- (void)viewDidLoad {
[super viewDidLoad];
rectangles = [[NSMutableArray alloc] init];
self.generalService = [GeneralServices sharedInstance];
//TODO: insert your own application keys
NSMutableDictionary *dict = [[NSMutableDictionary alloc] init];
// new way of authenticate. only need app key
[dict setValue:@"YOUR_APP_KEY" forKey:@"app_key"];
// old way of authentication which need access and secret key
// [dict setValue:@"YOUR_ACCESS_KEY" forKey:@"access_key"];
// [dict setValue:@"YOUR_SECRET_KEY" forKey:@"secret_key"];
[[CoreDataModel sharedInstance] insertApplication:dict];
}You can play around with our demo app to see how we build up the cool image search feature using ViSearch SDK.
In Xcode, go to File > New > Project Select the Single View Application.
Type a name for your project and press Next, here we use Demo as the project name.
First you need to install the CocoaPods Ruby gem:
[sudo] gem install cocoapods
pod setup
Then go to your project directory to create an empty Podfile
cd /path/to/Demo
pod init
Edit the Podfile as follow:
source 'https://github.com/CocoaPods/Specs.git'
platform :ios, '7.0'
...
pod 'ViSearch', '~>1.5.0'
...
Install the ViSearch SDK:
pod install
The Demo.xcworkspace project should be created.
You can also download the iOS ViSearch SDK directly. To use it, unzip it and drag ViSearch SDK folder into Demo project's file folder.
Then add it to your project
iOS 10 now requires user permission to access camera and photo library. If your app requires these access, please add description for NSCameraUsageDescription, NSPhotoLibraryUsageDescription in the Info.plist. More details can be found here.
ViSearch must be initialized with an appKey or accessKey/secretKey pair before it can be used.
#import <ViSearch/VisearchAPI.h>
...
// using default ViSearch client. The client, by default,
// connects to Visenze's server
// 1. new way of init ViSearch client with only app key
static NSString * const appKey = @"your_app_key";
[ViSearchAPI setupAppKey:appKey];
ViSearchClient *client = [ViSearch defaultClient];
// 2. OR old way of init ViSearch client with access and secret key
static NSString * const accessKey = @"your_access_key";
static NSString * const privateKey = @"your_secret_key";
[ViSearchAPI setupAccessKey:accessKey andSecretKey:secretKey];
ViSearchClient *client = [ViSearch defaultClient];
client.isAppKeyEnabled = NO;
...
// OR using customized client, which connects to your own server
static NSString * const privateKey = @"your_url";
ViSearchClient *client = [[ViSearchClient alloc] initWithBaseUrl:url
appKey:appKey];
...Please init ViSearch client in this way if you connect to another endpoint rather than default (https://visearch.visenze.com)
static NSString * const privateKey = @"your_url";
ViSearchClient *client = [[ViSearchClient alloc] initWithBaseUrl:url
appKey:appKey];
GET /search
Visually Similar Recommendations solution is used to search for visually similar images in the image database giving an indexed image’s unique identifier (im_name).
#import <ViSearch/VisearchAPI.h>
...
SearchParams *searchParams = [[SearchParams alloc] init];
searchParams.imName = @"imName-example";
[[ViSearchAPI defaultClient]
searchWithImageId:searchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];
...POST /uploadsearch
Search by image solution is used to search similar images by uploading an image or providing an image url. Image class is used to perform the image encoding and resizing. You should construct the Image object and pass it to UploadSearchParams to start a search.
- Using UIImage
#import <ViSearch/VisearchAPI.h>
...
UIImage *image = [UIImage imageNamed:@"example.jpg"];
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.imageFile = image
[[ViSearchAPI defaultClient]
searchWithImageData:uploadSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];- Alternatively, you can pass an image url directly to
uploadSearchParamsto start the search :
#import <ViSearch/VisearchAPI.h>
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.imageUrl = @"http://example.com/example.jpg";
[[ViSearchAPI defaultClient]
searchWithImageUrl:uploadSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];
...- Once uploading an image, you will receive a im_id attribute from the Search Results. If you want to search the same image again, you can save your bandwidth by specifying the im_id in the params:
#import <ViSearch/VisearchAPI.h>
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.imId = visearchResult.imId;
[[ViSearchAPI defaultClient]
searchWithImage:uploadSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];
...If the object you wish to search for takes up only a small portion of your image, or other irrelevant objects exists in the same image, chances are the search result could become inaccurate. Use the Box parameter to refine the search area of the image to improve accuracy. The box coordinated is set with respect to the original size of the uploading image. Note: the coordinate system uses pixel as unit instead of point.
// create the box to refine the area on the searching image
// Box(x1, y1, x2, y2) where (0,0) is the top-left corner
// of the image, (x1, y1) is the top-left corner of the box,
// and (x2, y2) is the bottom-right corner of the box.
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
Box *box = [[Box alloc]initWithX1:0 y1:0 x2:100 y2:100];
uploadSearchParams.box = box;
// start searching
...When performing upload search, you may notice the increased search latency with increased image file size. This is due to the increased time spent in network transferring your images to the ViSearch server, and the increased time for processing larger image files in ViSearch.
To reduce upload search latency, by default the uploadSearch method makes a copy of your image file and resizes the copy to 512x512 pixels if one of the original dimensions exceed 512 pixels. This is the optimized size to lower search latency while not sacrificing search accuracy for general use cases:
// by default, the max width of the image is set to 512px, quality is 0.97
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
// or you can explicitly set a param's settings
uploadSearchParams.settings = [ImageSettings defaultSettings];If your image contains fine details such as textile patterns and textures, you can use an image with larger size for search to get better search result:
// by default, the max width of the image is set to 512px, quality is 0.97
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
// set the image with high quality settings.
// Max width is 1024px, and the quality is 0.985. Note: Quality with 1.0 take hugespace
uploadSearchParams.settings = [ImageSettings highqualitySettings];Or, provide the customized resize settings. To make efficient use the of the memory and network bandwidth of mobile device, the maximum size is set at 1024 x 1024. Any image exceeds the limit will be resized to the limit:
//resize the image to 800 by 800 area using jpeg 0.9 quality
uploadSearchParams.settings = [[ImageSettings alloc]
initWithSize:CGSizeMake(800, 800) Quality:0.9];GET /colorsearch
Search by color solution is used to search images with similar color by providing a color code. The color code should be in Hexadecimal and passed to ColorSearchParams as a String.
#import <ViSearch/VisearchAPI.h>
...
ColorSearchParams *colorSearchParams = [[ColorSearchParams alloc] init];
colorSearchParams.color = @"012ACF";
[[ViSearchAPI defaultClient]
searchWithColor:colorSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];
...POST /discoversearch
Multiple Product Search solution is to search similar images by uploading an image or providing an image url, similar to Search by Image. Multiple Product Search is able to detect all objects in the image and return similar images for each at one time.
- Using UIImage
#import <ViSearch/VisearchAPI.h>
...
UIImage *image = [UIImage imageNamed:@"example.jpg"];
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.imageFile = image
[[ViSearchAPI defaultClient]
discoverSearchWithImageData:uploadSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];- Alternatively, you can pass an image url directly to
uploadSearchParamsto start the search :
#import <ViSearch/VisearchAPI.h>
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.imageUrl = @"http://example.com/example.jpg";
[[ViSearchAPI defaultClient]
discoverSearchWithImageUrl:uploadSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];
...- Once uploading an image, you will receive a im_id attribute from the Search Results. If you want to search the same image again, you can save your bandwidth by specifying the im_id in the params:
#import <ViSearch/VisearchAPI.h>
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.imId = visearchResult.imId;
[[ViSearchAPI defaultClient]
discoverSearchWithImage:uploadSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request succeeds
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Do something when request fails
}];
...After a successful search request, a list of results are passed to the callback function in the form of ViSearchResult. You can use following properties from the result to fulfill your own purpose.
| Name | Type | Description |
|---|---|---|
| success | BOOL | Is the request handled by the server successfully. Note: invalid parameters sent to the server can also make this property false. |
| error | ViSearchError | Besides the error caused by network condition, it also shows the error caused by invalid parameters sent to the server. |
| content | NSDictionary | The complete json data returned from the server.(This property may be deprecated in the feature) |
| imageResultsArray | NSArray | A list of image results returned from the server. |
| reqId | NSString | A request id which can be used for tracking. More details can be found in Section 7 |
| imId | NSString | An image id returned in the result which represents a image just uploaded. It can be re-used to do an upload search on the same image again. More details in Search by image |
You are encouraged to use the imageResultsArray, since content property may be deprecated in the future. Every image result is in the form of ImageResult. You can use following properties of a ImageResult to fulfill your own purpose.
| Name | Type | Description |
|---|---|---|
| im_name | NSString | the identify name of the image. |
| url | NSString | url of the image. |
| score | CGFloat | A float value ranging from 0.0 to 1.0. Refer to Section 6.3 Result Score. |
| metadataDictionary | NSDictionary | Other metadata returned from server. Refer to Section 6.1 Retrieving Metadata. |
// This is an example of image url search.
// The process of handling results by other kinds of search is similar.
[[ViSearchAPI defaultClient]
searchWithImageUrl:uploadSearchParams
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Iterate all returned results
for (ImageResult *result in data.imageResultsArray) {
NSLog("%@", result.url);//log result's image url.
//Do something here
}
} failure:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
NSLog("%@", data.error.message);//log network and server error message
}];You can provide pagination parameters to control the paging of the image search results. by configuring the basic search parameters BaseSearchParams. As the result is returned in a format of a list of images page by page, use setLimit to set the number of results per page, setPage to indicate the page number:
| Name | Type | Description |
|---|---|---|
| page | Integer | Optional parameter to specify the page of results. The first page of result is 1. Defaults to 1. |
| limit | Integer | Optional parameter to specify the result per page limit. Defaults to 10. |
// For example, when the server side has 60 items, the search operation will return
// the first 30 items with page = 1 and limit = 30. By changing the page to 2,
// the search will return the last 30 items.
...
SearchParams *searchParams = [[SearchParams alloc] init];
searchParams.page = 2;
searchParams.limit = 30;
// start searching
...To retrieve metadata of your search results, provide a list of metadata keys as the fl (field list) in the basic search property:
SearchParams *searchParams = [[SearchParams alloc] init];
searchParams.fl = @[@"price",@"brand",@"im_url"];To retrieve all metadata of your image results, specify get_all_fl parameter and set it to true:
SearchParams *searchParams = [[SearchParams alloc] init];
searchParams.getAllFl = true;In result callback you can read the metadata:
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
// Iterate all returned results
for (ImageResult *result in data.imageResultsArray) {
NSLog("%@", result.metadataDictionary);//log result's metadata.
NSLog("%@", [result.metadataDictionary
objectForKey;@"price"]);//log price in metadata
//Do something here
}
}Only metadata of type string, int, and float can be retrieved from ViSearch. Metadata of type text is not available for retrieval.
To filter search results based on metadata values, provide a map of metadata key to filter value as the fq (filter query) property:
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
// the type of "count" on db schema is int,
// so we can specify the value range, or do a value match
[uploadSearchParams.fq setObject:@"0, 199" forKey:@"count"];
[uploadSearchParams.fq setObject:@"199" forKey:@"count"];
// the type of "price" on db schema is float,
// so we can specify the value range, or do a value match
[uploadSearchParams.fq setObject:@"0.0, 199.0" forKey:@"price"];
[uploadSearchParams.fq setObject:@"15.0" forKey:@"price"];
// the type of "description" on db schema is string, so we can do a string match.
[uploadSearchParams.fq setObject:@"description" forKey:@"wooden"];
// start searching
...Querying syntax for each metadata type is listed in the following table:
| Type | FQ |
|---|---|
| string | Metadata value must be exactly matched with the query value, e.g. "Vintage Wingtips" would not match "vintage wingtips" or "vintage" |
| text | Metadata value will be indexed using full-text-search engine and supports fuzzy text matching, e.g. "A pair of high quality leather wingtips" would match any word in the phrase |
| int | Metadata value can be either:
|
| float | Metadata value can be either
|
ViSearch image search results are ranked in descending order i.e. from the highest scores to the lowest, ranging from 1.0 to 0.0. By default, the score for each result is not returned. You can turn on the score parameter to retrieve the scores for each image result:
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.score = YES; // result will include score for every image
// start searching
...If you need to restrict search results from a minimum score to a maximum score, specify the score_min and/or score_max parameters:
...
UploadSearchParams *uploadSearchParams = [[UploadSearchParams alloc] init];
uploadSearchParams.score = YES; // result will include score for every image
uplaodSearchParams.scoreMin = 0.3; // the minimum score is 0.3
uplaodSearchParams.scoreMax = 0.8; // the maximum score is 0.8
// start searching. Every image result will have a score within [0.3, 0.8].
...With Automatic Object Recognition, ViSearch /uploadsearch API is smart to detect the objects present in the query image and suggest the best matched product type to run the search on.
You can turn on the feature in upload search by setting the API parameter "detection=all". We are now able to detect various types of fashion items, including Top, Dress, Bottom, Shoe, Bag, Watch and Indian Ethnic Wear. The list is ever-expanding as we explore this feature for other categories.
Notice: This feature is currently available for fashion application type only. You will need to make sure your app type is configurated as "fashion" on ViSenze dashboard.
params.detection = @"all";You can use the Box parameter to restrict the image area [x1, y1, x2, y2] as the portion of your image to search for. When you input a box with 0 width and 0 height, eg. “box”:[574,224,574,224]. We will treat it as a point and detect the object over the current point.
You could also recognize objects from a paticular type on the uploaded query image through configuring the detection parameter to a specific product type as "detection={type}". Our API will run the search within that product type.
Sample request to detect bag in an uploaded image:
params.detection = @"bag";The detected product types are listed in product_types together with the match score and box area of the detected object. Multiple objects can be detected from the query image and they are ranked from the highest score to lowest. The full list of supported product types by our API will also be returned in product_types_list.
You can get the facet results by sending a list of fields to enable faceting on. Here are some limitations on the request:
-
Facet fields need to be marked as
searchableon ViSenze dashboard. Text field is not supported as facet field even it issearchable. System will return value range, the min, max value for numerical fields which are in ‘int’, ‘float’ type. -
Only facet values that exist in current search results will be returned. For example, if your search results contain 10 unique brands, then the facet filters will return the value for these 10 brands.
-
Facet value list is ordered by the item count descendingly. When the value is set to all (facets = *), all the searchable fields will be used as facet fields.
| Name | Type | Description |
|---|---|---|
| facets | array | List of fields to enable faceting. |
| facets_limit | Int | Limit of the number of facet values to be returned. Only for non-numerical fields. |
| facets_show_count | Boolean | Option to show the facets count in the response. |
searchParams.facets = @[@"brand" , @"price"];
searchParams.facetShowCount = YES;
searchParams.facetsLimit = 10;
// view facet results
ViSearchResult.facets
To improve search performance and gain useful data insights, it is recommended to send user interactions (actions) with our visual search results. Our tracking SDK (Swift) is located at https://github.com/visenze/visenze-tracking-swift.
You can initiliase ViSenze tracker with a tracking ID (code) by logging to ViSenze dashboard. There are two different endpoints for tracker (1 for China and another for the rest of the world). If the SDK is intended to be used outside of China, please set forCn parameter to false
import ViSenzeAnalytics
...
let tracker = ViSenzeAnalytics.sharedInstance.newTracker(code: "your-code", forCn: false)Currently we support the following event actions: click, view, product_click, product_view, add_to_cart, and transaction. The action parameter can be an arbitrary string and custom events can be sent.
To send events, first retrieve the search query ID found in the search results call back:
success:^(NSInteger statusCode, ViSearchResult *data, NSError *error) {
NSString* queryId = [data reqId];
Then the linked events can be sent as follows:
# send product click
let productClickEvent = VaEvent.newProductClickEvent(queryId: "ViSearch reqid in API response", pid: "product ID", imgUrl: "product image URL", pos: 3)
tracker.sendEvent(productClickEvent) { (eventResponse, networkError) in
}
# send product impression
let impressionEvent = VaEvent.newProductImpressionEvent(queryId: "ViSearch reqid in API response", pid: "product ID", imgUrl: "product image URL", pos: 3)
tracker.sendEvent(impressionEvent)
# send Transaction event e.g order purchase of $300
let transEvent = VaEvent.newTransactionEvent(queryId: "xxx", transactionId:"your trans id", value: 300)
tracker.sendEvent(transEvent)
# send Add to Cart Event
let add2Cart = VaEvent.newAdd2CartEvent(queryId: "ViSearch reqid in API response", pid: "product ID", imgUrl: "product image URL", pos: 3)
tracker.sendEvent(add2Cart)
Below are the brief description for various parameters:
| Field | Description | Required |
|---|---|---|
| queryId | The request id of the search request. This reqid can be obtained from the search response handler:ViSearchResult .reqId |
Yes |
| action | Event action. Currently we support the following event actions: click, view, product_click, product_view, add_to_cart, and transaction. |
Yes |
| pid | Product ID ( generally this is the im_name) for this product. Can be retrieved via ImageResult.im_name |
Required for product view, product click and add to cart events |
| imgUrl | Image URL ( generally this is the im_url) for this product. Can be retrieved via ImageResult.url |
Required for product view, product click and add to cart events |
| pos | Position of the product in Search Results e.g. click position/ view position. Note that this start from 1 , not 0. | Required for product view, product click and add to cart events |
| transactionId | Transaction ID | Required for transaction event. |
| value | Transaction value e.g. order value | Required for transaction event. |