If you're working with bigger data -- make sure you're shipping your pairs out to each thread, and not the whole data frame. That overhead to copy a huge df across threads is a killer. But otherwise, yeah, unfortunately this is the way
anyway, point being, this way madness lies 😂 4/4
All of this is single threaded, because the overhead of moving large objects across threads was too high... I've fully considered writing the data into DuckDB and using the spatial extension, to hopefully get better parallelism, but haven't had the focus on this project for it
and then scan the matrix for distances below a certain threshold. For my needs, I wind up doing a single st_distance call and then processing the matrix in C++ for an ounce more performance. It's gross. One of the funniest comments I've written though, imo: github.com/tidymodels/s...
Create and summarize spatial resampling objects 🗺. Contribute to tidymodels/spatialsample development by creating an account on GitHub.
optimizing this is the most complicated part of spatialsample. The best solution I've found is either parallelizing it yourself (make a list that's like c(1, 2), c(1, 3) etc and use it to index the data frame, then run st_intersects for each pair separately) or compute a single distance matrix 1/
Alternatives to the traffic light colour scheme 🟢🟡🔴 Your dashboards will look fresh, will be more accessible and your clients will think you're a magician. 📊 👇
Never once did it occur to me that function had a serious use case
This is USGS water gage data. The black dots are "all-time high for this day." The pink outlined ones are above flood stage, dark blue are "much above normal." These stretch across a thousand miles.This is USGS water gage data. The black dots are "all-time high for this day." The pink outlined ones are above flood stage, dark blue are "much above normal." These stretch across a thousand miles.
That said, the online resources around it -- like Michelle's workshops!! -- have gotten DRASTICALLY better in the amount of time I've been using it