Wow, this is so cool!
That's really cool. Thank you
It would be really interesting to have data about that. But I think for incremental pipelines running over months or years, the amount of data that is inserted or updated regularly is much smaller than the amount of data that stays unchanged.
I think these could be the design principles for a new era of programming. Avoid creating 200 lines of boiler plate code just because you created a simple class.
Are you sure you need a garbage collector? If you use Arc and RC sparingly and don't introduce any circular dependencies then the memory should free itself.
That's so awesome. I can't wait. And I'm really curious what changed from the first edition. Some technologies seem to never change while others are progressing rapidly.
Chris Lattner is awesome. It's just that Mojo tries to be Swift, Rust, Python and Cuda at the same time. And it doesn't do it by providing a unified abstraction but by bundling all capabilities together, like C++. You can program like Cuda or Python but it feels like using 2 different languages.
I think Mojo is trying to do too much. It reminds me of a quote from Ken Thompson about C++: [C++] certainly has its good points. But by and large I think it's a bad language. It does a lot of things half well and it’s just a garbage heap of ideas that are mutually exclusive.
With Arcs you would still need to think about shared mutability. And it would probably lead to quite a performance hit. The atomic operations force the compiler to order the instructions a certain way. I think the ownership system is actually a nice abstraction of how to think about your values.
If you want to ingest a Kafka stream into an iceberg table in rust. I've been playing around with this repo: github.com/dashbook/kaf.... I haven't updated it in a while and it's a bit opinionated with the schema registry and the message format. But it could provide a starting point.
Contribute to dashbook/kafka-iceberg-ingest development by creating an account on GitHub.