photog.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
A place for your photos and banter. Photog first is our motto Please refer to the site rules before posting.

Administered by:

Server stats:

242
active users

#dataflow

0 posts0 participants0 posts today

I'm not feeling the warm fuzzies about trying to do GCP Dataflow (aka Apache Beam) with Typescript. Doesn't feel fully baked.

Also doesn't feel like a lot of doc/examples nor many people using it. I don't think our org should be on the bleeding edge of something like this for a core piece of our infrastructure.

Dunno. Will keep working on it for a bit.

For many years I've observed the following in parallel:

1. General confusion (and in some cases disdain) about the notion of a #SemanticWeb

2. Steadfast effort by a community to keep the vision going, stealthily, as exemplified by the #LODCloud (#DBpedia,, #Wikidata, #Uniprot etc..) and Schema.org (#SchemaOrg)

3. #DataSilos and all the impedance they inflict on the agility of individuals and enterprises alike on the rise i.e., #DataAccess and #DataFlow are still challenging, unnecessarily.

One of the things I'll be talking about in the stream tomorrow: Understanding that thi.ng/rstream constructs are essentially defining a #dataflow #graph much like you'd know from a node-graph based editor (a la Blender/Houdini etc.), only obviously with the difference of actually creating such a graph programmatically rather than manually via visual editing. Personally I find this programmatic approach much more powerful, but a visual representation makes it of course much easier to debug & analyze such graph topologies, also to explain & illustrate to others how data and state changes are flowing through the app/system and/or where to attach new features/nodes or optimize/restructure the topology...

Another massive benefit: This approach makes it much easier to build features such that only the most minimal amount of work needs to be done, helping to achieve better performance...

The attached diagram was generated with #Graphviz via thi.ng/rstream-dot and shows the dataflow graph for the image vectorization tool mentioned in previous toots.

Full size version:
demo.thi.ng/umbrella/trace-bit

The tool itself:
demo.thi.ng/umbrella/trace-bit

The second image shows the minimal code required to generate the #visualization source file...