How Hip are Brooklyn Neighborhoods?
IBM Data Science Capstone Project
Introduction
Brooklyn NY is notoriously (sometimes comically) hip — known for hot yoga studios, elusive speakeasies and overpriced coffee. But Brooklyn is a big place with a myriad of different neighborhoods. For this project, I explored how we can use data to identify which neighborhoods in Brooklyn offer the highest number of Hipster-amenities.
The output from this project could be used by prospective renters looking for an archetypical Brooklyn experience, or real estate agents hoping to most effectively promote properties in these areas. Personally, as a hipster who has lived in Brooklyn for the past 7 years, I want to use data to see if I can validate some existing stereotypes and understand, empirically, which neighborhoods are the hippest in Brooklyn.
Data Sources
To undertake this investigation, I leveraged two data sources:
- NYC Neighborhood Data. ‘newyork_data.json’ provides a comprehensive list of neighborhoods in New York that includes their Boroughs and Latitude and Longitude.
- The Foursquare API. I can pull information about adjacent venues from ‘api.foursquare.com’.
The following analysis was completed in Python on a Jupyter Notebook using the pandas, numpy, geopy, seaborn, and matplotlib libraries as well as folium for visualizing geospatial data. You can download the notebook here to view the original code.
The NYC Neighborhood Data from ‘newyork_data.json’ contains data on all neighborhoods in New York. Since I will only be looking at Brooklyn, I started by dropping neighborhoods in all other boroughs.
Before moving into exploratory analysis, I mapped the cleaned Brooklyn data using folium to ensure it was correct formatted.
Lastly, I connected my notebook to the Foursquare API to pull relevant data about venues in Brooklyn. Foursquare’s powerful API contains information like venue addresses, venue category, customer rating and reviews.
Methodology
With data from Foursquare and our NYC database imported and cleaned up, I started exploratory analysis. This would include identifying the neighborhoods with the highest number of hipster amenities, ranking them, and comparing them to the average Brooklyn neighborhood to gauge significance. To do this, I started by creating a function that used the Foursquare API to identify how many venues were near each neighborhood in Brooklyn based on their recorded lat/long position.
I ran that function on all neighborhoods in Brooklyn, then create a dataframe counting the number of venues in each neighborhood per Venue Category (e.g. Bank, Coffee Shop, Library, etc…). To understand which neighborhoods have the most venues overall, I add a ‘TOTAL VENUES’ column to this dataframe that aggregates across venue categories.
To understand which neighborhoods are particularly hip, I needed to differentiate between standard venues (places a normie would go) and venues that appeal to hipsters. I created a new dataframe that only included hipster venues (e.g. Yoga Studios, Cafes, Indy Movie Theaters, and Speakeasies). While this list is not comprehensive, it represents some of the highest hipster priorities, and should give a fair assessment as to how hipster an area might be. Further research into hipster culture could and should be completed in order to make this list as representative as possible.
Ordering the neighborhoods by the value of their total number of hip venues, I created a list of 10 neighborhoods that offer the highest number of hipster amenities in Brooklyn. A simple stacked bar chart shows the breakdown of those hipster amenities across those 10 neighborhoods to provide a breakdown of how these neighborhoods compare.
While some neighborhoods have a high number of Hipster venues, this could be the result of simply having a larger number of total venues. A quick regression plot showed which neighborhoods have a disproportionate number of hipster venues compared to their total number of amenities.
As you can see, neighborhoods like Bay Ridge and even Downtown Brooklyn have the highest number of venues overall, but not the highest number of hip venues. Whereas neighborhoods like Greenpoint, Northside and Bed Stuy a proportionately a high number of hipster venues than other neighborhoods with a similar number of overall venues (Bed Stuy has nearly double the number of hipster venues than Vinegar Hill despite having a similar number of overall venues). This indicates that, Downtown Brooklyn’s hip-ness might be diluted with non-hip amenities when compared to a neighborhood like Greenpoint, which is dense with venues that would appeal to hipsters.
I combined the hip venue total and overall venue total back into the neighborhood dataframe in order to create a final map with folium that visualizes the total number of venues in each neighborhood and highlights the total number of hip venues through a color gradient. The size of each bubble represents the total number of venues while the shade of blue indicates how many hip venues are in that neighborhood. Darker blue means more hipster venues.
Results
Immediately, the map makes it clear that the “hip” parts of Brooklyn are actually quite clustered immediately adjacent to Manhattan. The majority of Brooklyn neighborhoods do not actually offer a significant number of hipster amenities.
From the raw ranking, we can callout the following neighborhoods as possessing a high number of hipster venues:
1. North Side (Williamsburg), 18 hip spots out of 100 venues
2. Greenpoint, 17 hip spots out of 100 venues
3. Cobble Hill, 14 hip spots out of 98 venues
4. Carroll Gardens, 13 hip spots out of 100 venues
5. Fort Greene, 11 hip spots out of 67 venues
6. Boerum Hill, 11 hip spots out of 86 venues
7. Prospect Heights, 11 hip spots out of 78 venues
8. South Side (Williamsburg), 11 hip spots out of 100 venues
9. Brooklyn Heights, 10 hip spots out of 100 venues
10. Downtown Brooklyn, 9 hip spots out of 100 venues
Looking at our regression line, however, we might exclude Downtown Brooklyn, Brooklyn Heights and South Side Williamsburg from this list as their ratio of hip venues to total venues is not as favorable as the other neighborhoods on our top 10 list.
For those interested in moving to particularly hip parts of Brooklyn, our data would indicate that North Side Williamsburg or Greenpoint might be your best option, followed by Cobble Hill, Carroll Gardens, Fort Greene, Boerum Hill and even Prospect Heights. Worth mentioning, while Bed Stuy only offers 28 total venues (from this Foursquare data), 1 in 4 of those 28 is a hipster-approved amenity–so if you only need coffee and craft beers, Bed Stuy might be for you.
Conclusion
While this analysis is cousory and (obviously) a bit tongue-in-cheek, it could be interesting and genuinely valuable to overlay this type of data with average costs of apartments in the area to get an understanding of which hip areas are most affordable. When analyzed over time, this type of analysis could also reveal how hipster-influence impacts gentrification.
Again, if you are interested in checking out the full code, you can download it here. Thanks for your time and I hope you enjoyed this data project!
I ❤ Brooklyn
