If the video doesn't load, then visit this link here
Article Navigation
In this article:
Overview
A/B Testing Section:
The 'A/B Testing' section of the 'Rankings Hub' allows you to test & verify the impact of listing optimization changes you make to your Airbnb listing.
You can A/B test your first photo, the entire photo arrangement, listing title, description, policies & more.
Changes you make to your listing will positively, negatively, or neutrally impact your listing's search visibility.
Rankbreeze automatically tracks the changes a guest sees on your listing page (ie. your content & policy settings) & will automatically measure their impact. You can also track changes manually too.
Auto-Tracked Changes:
Rankbreeze automatically tracks new content & policy changes of your Airbnb listings. Therefore if you test out a new photo arrangement, we'll automatically log it and run an A/B test for 14 days (Rankbreeze automatically measures A/B tests for 14 days by default but this duration can be changed in the Settings page)
For a list of all the changes automatically tracked, please see our Auto-Tracked Changes guide
How to start?
All you have to do is make a change, such as adjusting your title, and Rankbreeze will log it as a change the next day.
Manually Tracking Changes:
There will be times when you want to test something outside of our list of auto-tracked changes. This might involve changing your host profile details, implementing a new review-response process, or anything your mind can dream of.
When you want to test a change that's outside of our auto-tracked list, then run a Manual A/B test.
How to start?
Start by pressing the “Track a Change” button. A pop-up window will open and from here you can enter your change & select a testing duration period. We recommend testing each change for 14 days.
Why It Matters
Measuring your listing changes is crucial because it provides evidence of changes that work and don’t work for your listing. This enables you to make informed decisions and improve your listing’s visibility & performance. This data-informed approach helps you reach more guest's searching on Airbnb quicker.
Step-by-Step Walkthrough
Step 1: Access the A/B Testing section
Log in to your Rankbreeze account. Navigate to the “My Properties” page and click on the “Rankings” button. Scroll down to the A/B Testing section. It is the fourth section on this page.
Step 2: Implement a Test On Your Listing
There are generally 3 types of changes you can make to your Airbnb listing: content, pricing, & policies.
Auto-tracked changes:
If that change you made was something that a guest would see on your listing page, then Rankbreeze would automatically log the change for you the next day & start running a 14 day A/B test.
Here's a full list of the changes we auto-track
Manually Tracked changes:
Click the Track a Change button to enter in a change manually & to start tracking its impact on your search visibility.
Describe the change to your listing in the space provided.
Click the slider and drag the icon to choose how long you want the test to run. This is called the Duration Period.
? The infinity duration can be used for updates that won’t be reverted e.g. renovations.
Click "Track change" to begin the test.
Step 3: Review Results
Rankbreeze measures the performance of your listing against Similar Listings.
Similar Listings are created & defined by Airbnb (link here)
By default, you'll see the following four metrics to see your progress in:
- Average City Rankings
- Impressions
- Click-through rate
- Booking rate
Here are the definitions of all the AB Testing metrics
Pro Tips:
You can also measure Listing Views & Conversion Rates. These are hidden by default to make the interface simpler but you can turn it on by clicking the Settings icon 
You don't need to wait until the end of your test to start seeing results. During the middle of your test, you'll start to see preliminary results showing up.
Click on any of the metrics to inspect a specific metric. You can also click the Details button.
The Daily Chart
From this view, you can tell how your listing is progressing against Similar Listings on each day of your test.
From the start date of the test, Rankbreeze will record your listing's initial data & measure the progress of your metrics each day. It will do the same for Similar Listings and measure the difference in your listing's progress against Similar Listings.
The purple line represents your listing and the grey line represents Similar Listings. In the example above, your listing is outperforming at a faster pace than Similar Listings, which represents a positive impact.
The Monthly Chart
If you click the Monthly tab, you will see data divided into the next upcoming months.
Each month represents searches that guests are making in that specific month.
For example, when it comes to Monthly impressions in July, those are the number of times you appeared on the search results page for guests searching for stays in July.
Each month will also show you the change in that metric & whether it went up or down.
For example, let's focus on the chart below & the month of July: This listing's impressions have increased by 711 impressions since the beginning of the test. However, Similar Listings have increased by 1000 impressions.
This is an indicator that guests are searching for July accommodations but you are not increasing at a similar pace to competitors so you should inspect your listing's content, offers, promotions & prices specifically for July dates. July is also the only month the property has underperformed against Similar Listings.
Pro Tips
- Changes are displayed in the Property Search Rankings Chart: When you track a change in the Optimization Journal, or an auto-tracked change is logged, it will show up as a dotted red line on the Property Search Rankings chart. Use this visual to analyze the trend of your rankings over time and your listing optimizations over time.
Conflicting Changes: In case you make multiple changes within a currently running test, then Rankbreeze will tell you the conflicting changes so you're aware of the potential conflicts. For example, you might have an AB test running for 14 days where you're measuring the impact of a cover photo change. Then 5 days into the test, you change your title. Rankbreeze will show you the conflicting changes within the cover photo test to inform you of the conflicting title change.
Testing Guidelines
- Find Gaps in Your Search Booking Funnel: The default metrics Rankbreeze sets up for you represent the typical search path guests go through to book a property.
- First, they see you listing on the Airbnb search results page (rankings & impressions),
- then, if your listing is worthwhile, they click on your listing to review it closer (click-through rate)
- then, after comparing your listing against others, they decide to book your property (booking rate)
When examining your metrics, you might see one of your metrics underperforming against your other metrics, this means you have a hole in your search booking funnel & it would be best to fix that.
In this example, you can see the hole being impressions.
Pick One Thing to Test at a Time: To know the impact of a change you're making, change one thing at a time. If you test multiple changes at a time, you probably won't be able to attribute the impact to a specific change.
Determine Duration: We recommend tracking changes for at least two weeks. This gives Airbnb enough time to collect data and adjust your rankings based on it.
Run the Test(s): Implement the change on Airbnb and run the test.
If you're deciding between two options, like an aerial shot or a balcony view for your cover photo, follow these steps:Make the first change (e.g., use the aerial shot) and run the test.
After the first test is complete, implement the second change (e.g., use the balcony view) and run the test again.
Avoid stopping the test early based on initial results.
Analyze the Results: Determine if the change helped achieve your goal.
If you’re testing different options, decide which option performed better.
Make a Decision: Once you’ve analyzed the results, decide whether to revert your change or update your listing with the option that performed better.
Test Seasonally: Leverage A/B testing during different seasons to understand how seasonal variations affect guest preferences and optimize your listings accordingly.
FAQs
Q: How long should I track a change?
A: It depends on the changes. For irreversible changes (e.g., remodeling), you can add a manual change set to a duration of 'infinity' since they can't be undone. For all other testing, we recommend tracking changes for at least two weeks. This gives Airbnb enough time to collect data and adjust your rankings based on it.
Q: What kind of changes should I track?
A: Track any type of change that could have an impact on your listing visibility such as policies, cover photo choice, listing titles, pricing strategies, etc.
Q: Can I track multiple changes at a time?
A: Yes. However, if there is a significant impact on your listing, you probably won't be able to attribute the impact to a specific change. Therefore, we recommend testing one change at a time.
Q: Can I access the results of tests I did in the past?
A: Every entry you add to the AB Testing section is kept until you delete it. You can always go back to review changes you made in the past.