The Idea
We, Oriol Batalla and Andrea Estalayo, are a curious Engineer and a innovative Interior Architect that were looking to move to a bigger city. We were very happy and excited with that idea. But we realized that the coolest cities also included a extra premium on their rents.
To achieve this objective we considered to offset the rent using Airbnb to rent a spare room on their future apartment. But the lack of statistics made this a risky decision. So minimize this risk, we started to collect data from Airbnb with a simple script. This simple script started to grow and evolved into a complete system capable of crawl regularly almost all room in Airbnb for an entire city.
That is the little story behind how we ended up building Airbnb Stats: a site that can provide some statistics about the current listings, including location of the most profitable areas or daily average revenue per city.
Current Technology
Back-end
- Python brain that runs continuously grabbing data from Airbnb updating the front end every few days.
- MongoDB as a powerhouse to store the data with currently more than 15 Gb and 1M listings so far.
- Proxy libraries and own made lists to avoid Airbnb getting upset 😉
- Microsoft Azure services based 24/7 scraping.
Front=end
- Mapbox to display and render the geo-data. Perfect to display big amount of data.
- Charts use Highcharts library which includes some cool features that helps on automation
- Simple blog based on WordPress to track features and updates
- Front-End is separated from the Back-end to maintain anonymity of the scrapper.