RESO Web API Replication

If you'd like to replicate data but do not have an API key with replication permission, or if you find a problem with this outline, please contact api-support@fbsdata.com.

1. Overview
2. Step 1. Initial Download and Pagination
3. Step 2. Updating Data
4. Step 3. Purging Stale Data
5. Best Practices

Overview

Replication API Keys

Developers who wish to replicate data need to use an API key with replication access (now the default). API keys with permission to replicate must hit a separate replication endpoint: https://replication.sparkapi.com/Reso/OData. Requests from API keys with replication access will fail if they are made to the regular RESO Web API endpoint.

To make replicating data easier, these keys also have the following privileges:

• The maximum $top value is raised to 1000 records
• The Member and Office resources do not apply company or office restrictions when searching user records for IDX roles

Replication API keys are otherwise identical to regular API keys, with their access to resources and data governed by the API key's role.

The Replication Process

When replicating records from any resource, there are three steps:

1. The initial data download
2. Periodically checking for changes and updating data
3. Purging stale data

While the examples below demonstrate how to duplicate from the Property resource, the same basic approach is appicable for replicating Member and Office records.

Step 1. Initial Download and Pagination

To kick off the initial download, you'll need to make a request to the Property endpoint. If you don't include a $filter parameter with the request, this will pull results from all listings accessible with your API key, and you'll be able to paginate through the results.

GET https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$count=true

The $top parameter determines the number of records you'll get back per page. The maximum value is 1000, but if you encounter timeouts or errors, try using a lower $top value. You may also wish to include a $count=true parameter, which will include an @odata.count field in the response that indicates the number of records matching your request.

Paginating through the full result set can be accomplished one of two ways:

1. By using the @odata.nextLink values from the API response to make your next request (preferred)
2. By incrementing the $skip parameter with each request

Using the @odata.nextLink

When a RESO Web API response doesn't return all matching results on a single page, the response will include an @odata.nextLink. That contains a RESO Web API request to return the next page of results, which is accomplished with a $skiptoken parameter set to the ListingKey of the last record on the current page.

"@odata.nextLink": "https://replication.sparkapi.com/Reso/OData/Property?%24skiptoken=20000612235426456684000000&%24top=1000",

Once you've finished processing the current page of results, you can use that value for your next API request. Once you've reached the last page of records, the @odata.nextLink won't be returned in the RESO Web API response.

Using the @odata.nextLink is the recommended method for replicating Property records. By default, results are ordered by ListingKey, so using the @odata.nextLinks ensures the order of the results is consistent as you paginate through them.

Please note that the @odata.nextLink method is not implemented for the Member and Office resources, and the $skip parameter should be used.

Incrementing the $skip parameter (use with caution)

An alternative method of paginating through results is using the $skip parameter. This specifies a number of records to skip before returning results. Use this method with caution; see below.

If this is your first request...

GET https://replication.sparkapi.com/Reso/OData/Property?$top=1000

...then your second request should include a $skip=1000 parameter...

GET https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$skip=1000

...and so on. Increment your $skip value by whatever value you are using for the $top parameter. Note that this differs from the $skiptoken parameter, which takes a ListingKey as a value and skips all records ahead of that listing.

Step 2. Updating Data

Keeping your local data updated involves making periodic requests for recently updated listings, which is done by using the ModificationTimestamp field with the $filter parameter.

We recommend polling for updated records no less than once every hour. to ensure your data reflects recent changes.

To request listings that have been modified within a given window, use a $filter that includes both an lower bound - ModificationTimestamp gt <timestamp> - and an upper bound: ModificationTimestamp lt <timestamp>. Here's an example request:

GET https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$filter=(ModificationTimestamp gt 2021-04-12T08:10:00Z and ModificationTimestamp lt 2021-04-12T08:20:00Z)

Note that parentheses are required if you include additional filter terms.

Use the more recent timestamp from the previous request as the lower bound for the next request.

The same pagination methods outlined above should also be used to paginate over the updated records; the @odata.nextLink method is recommended.

If you use Media, Room, Unit and Open House data, we recommend updating these for each property as well.

Step 3. Purging Stale Data

Purging stale data is the final part of the replication process. It's common to lose access to listings that were previously accessible with your API key. This most frequently happens when a listing changes status, and with IDX API keys, though it can happen with API keys of any role. Consequently, it's necessary to have a process to check whether listings in your local data can still be accessed.

This process should be run no less than once daily, and MLS-specific rules may require more frequent updates. Failure to do so can result in continuing to display listings that are no longer active or eligible for display.

We recommend using a $select=ListingKey parameter to limit the data you get back, then paginate through the listings results as outlined above in step 1 (again, the @odata.nextLink method is recommended). Compare all ListingKey values returned with the values in your local data, and purge any records that do not match.

GET https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$select=ListingKey

Best Practices

Displaying Agent & Office Data on Listings

We strongly recommend data consumers:

1. Display listing and buyer agent and office data from Member and Office records rather than from the Property payloads themselves.
2. Replicate and periodically update the Member and Office resources to ensure they remain up-to-date.

While the API response from the Property resource returns listing and buyer agent and office fields (e.g. ListAgentPreferredPhone) per the RESO Data Dictionary, agent and office data is pulled in from the Member and Office records at the time of the request; it is not stored as part of the actual property records. Changes to Member and Office records do not change Property record modification timestamps.

The following fields from the Property response contain keys for the corresponding Member and Office records:

• ListAgentKey
• ListOfficeKey
• CoListAgentKey
• CoListOfficeKey
• BuyerAgentKey
• BuyerOfficeKey
• CoBuyerAgentKey
• CoBuyerOfficeKey

For example, if a Property record returns this ListAgentKey...

"ListAgentKey": 20140227182254339191000000,

...then the following request will return the Member record with the data that should be displayed:

GET https://replication.sparkapi.com/Reso/OData/Member('20140227182254339191000000')

Replication Optimizations

We recommend the following best practices to ensure efficient data replication and data accuracy.

The $select parameter: Use the $select parameter with your requests to limit the fields you receive back to only those which you actually use. Doing so can cut out large amounts of data you don't need.

Media, Room, Unit and Open House expansions: If you use Media, Room, Unit or Open House data, you can request it be included along with the standard response from the Property resource, by including any of the Media, Room, Unit, and Open House expansions with your request:

GET https://replication.sparkapi.com/Reso/OData/Property?$top=1000&$expand=Media,Unit,Room,OpenHouse

Alternatively, you can retrieve this data with separate requests to property-specific endpoints (e.g. /Property/<ListingKey>/Media). If you do so, we strongly encourage you to make use of the PhotosChangeTimestamp, VideosChangeTimestamp and DocumentsChangeTimestamp fields to see if any media needs to be updated prior to polling for that data.