Composable information

Bytes of information about programming and the web.

[peterhil]

Spellbook Browser extension

These are my latest notes about how to implement a bookmark manager browser extension. Maybe there is some interesting information for somebody thinking of implementing something similar.

The screenshots are from actual simple extension that I have developed in the past four days using a previous unfinished prototype.

It shows a list of filtered bookmark categories when you start to type in a category name, but there is still some usability problems.

This is implemented using Vue.js without Redux or Vuex, and uses Semantic UI as a CSS framework. I use RxJS version 6, even though it is a huge library even when trying to slim the build down using webpack.

I switched to Semantic UI from Element UI because the latter wrote styles with Javascript and used negative margins, and I see both as huge warning signs for not using that CSS framework!

The extension will be a quite simple one for now – it will not use a graph database or have syncing features.

I just want to have and use something better than the current Material UI bookmark manager that came with Google Chrome version 69, because UI wise it is not good for handling the 15.000 bookmarks and over thousand categories of bookmarks that I have.

Implementation plan for the bookmark manager

3.7.2018

In order to get something releasable done, this is the plan:

Use Redux and implement browser extension using some JS tech – if this can be used, then the bookmarks can be easily stored in IndexedDB.

JS tech

Implement a browser extension using:

Vue.js (React may make more sense with a reactive library)

Redux (is it really needed?)

Most.js, RxJS v6, Kefir, Highland.js or Bacon.js? (see perf comparison by Most)

maybe Ocaml/Reason, ClojureScript or at least ES6+?

Utility libraries:

Ramda

Nanoid

So options are:

React, Redux and Most.js with redux-most (See article 2 below)

Vue.js, Redux and Most.js with redux-vue or Vuedeux and redux-most?

Redux-observable, Redux, Vue/React ...

Redux and/or RxJS

Browser extensions operate in a very asynchronous manner, so maybe using RxJS would be a good idea?

Google Search: Redux and RxJS

Async actions in Redux with RxJS and Redux Observable

How to make your React app fully functional, fully reactive, and able to handle all those crazy… React + Redux + React-observable/react-cycles (comparable to redux-most!)

Example code:

https://dev.to/andrejnaumovski/async-actions-in-redux-with-rxjs-and--redux-observable-efg

Article https://ivanjov.com/reactive-redux-state-with-rxjs/ and Github https://github.com/IvanJov/react-rxjs-github-search

Alternative languages and tech

Vue vs React

comment on How and why we moved to Vue.js

Against using RxJS

RxJS: An object lesson in terrible good software | Isotoma: Our blog

On the other hand, see this comment on Why isn't RxJS more popular? Are there bad parts about it that's slowing adoption? : javascript ”One of the big problems with RxJS has actually been addressed in version 5, which is that it's really hard to debug. Version 4 uses a trampoline architecture, which essentially knocks over the call stack, and renders them pretty useless. You're also usually super deep into library code, which is not where you want to be. Version 5 uses a different approach, and is supposed to make the debugging situation much better.”

Why isn't RxJS more popular? Are there bad parts about it that's slowing adoption? : javascript

Unredux

This repo features a number of realistic apps built without (frontend) frameworks. We use a dataflow similar to CycleJS built on Kefir and React.

We weren't happy with clunky imperative architectures Redux, VueJS, MobX, and GraphQL apps end up with, so we decided to codify our own approach. Like CycleJS, it's 100% reactive (declarative in time!). Unlike CycleJS, it doesn't hide side-effects in drivers. See docs for more details about our motivations and design decisions.

Unredux

Unredux post edit example

ClojureScript (and core.sync like async/await in ES6)

Why I moved from RxJS to ClojureScript

Promises and Async/await in ES6+

Even with async/await, raw promises are still key to writing optimal concurrent javascript

Storage

Storage consists of:

Bookmarks

Categories

Category structure

Bookmark categories

Redux

Think and experiment about caching/indexes, read Using the Redux Store Like a Database – Hacker Noon!

Immediately after getting first version working, use large dataset to find out performance problems early!

IndexedDB

Mozilla

Google documentation

Chrome storage sync (asynchronous bulk read and write) has some severe limitations, however an extension can have unlimitedStorage permission for local storage (synchronous). Maybe another way to sync is needed?

IndexedDB with Redux

redux indexeddb - Startpage Web Search

prateekbh/redux-storage-engine-indexed-db: indexedDb engine for redux-storage

react-stack/redux-storage: Persistence layer for redux with flexible backends

Using the Redux Store Like a Database – Hacker Noon

reduxjs/reselect: Selector library for Redux

Normalizer | Redux without Profanity

javascript - How to integrate Redux with very large data-sets and IndexedDB - Stack Overflow

Best Practices for Using IndexedDB  |  Web Fundamentals  |  Google Developers

IndexedDB with promises and async/await – Filip Vitas – Medium

Offline-First Web Application Using Redux and PouchDB

Open questions

Validation

Validation on Redux store (and when syncing)?

Caching bookmark categories

into Redux store?

using binary trees?

Data synchronisation

How to synchronise bookmarks between different browsers?

GUN database could be one option, even though it’s graph API is not very developer friendly. But using that would spare me writing a CRDT algorithm.

Conflict free replicated data types (CRDTs)

Practical Demystification of CRDTs - Speaker Deck orbitdb/crdts: A library of Conflict-Free Replicated Data Types for JavaScript

dominictarr/crdt: Commutative Replicated Data Types for easy collaborative/distributed systems.

orbitdb/orbit-db: Peer-to-Peer Databases for the Decentralized Web

pfrazee/crdt_notes

Conflict-free Replicated Data Types

A Conflict-Free Replicated JSON Datatype | Hacker News

1201.1784.pdf

Conflict-free Replicated Data Types

Home - Postman Documentation

PsychoLlama/graph-crdt: Commutative graphs made for real-time, offline-tolerant replication

Automerge: JSON-like data structure for building collaborative apps | Hacker News

[peterhil]

Journey in making a DockerFile

Introduction

Yo Docker! I heard you make virtualization simple, so I virtualized while virtualizing to make things even more amazing! I made it my way!

Seriously, I needed to write a Dockerfile as a part of technical task of an interview process. Here is my journey of the very easy process of setting up docker development environment on my perfectly good desktop iMac (8 GB ram, 8 cores, 1 TB SSD drive and a beautiful 27 inch screen) from the year 2010.

start time: about 15 o'clock

First try on my Mac

Brew install Docker

Find out that this might not be enough

Download latest Docker.app desktop application and copy it over the old version (wait 10 minutes for download)

Find out that my home desktop iMac's CPU does not have virtualization capabilities as it is from 2010, just before supported the machines

Download the older Docker toolkit (wait 10 minutes for download)

The Docker toolkit does not work on my Mac either

Detour with the old Linux laptop

Try to install docker-ce on my Linux laptop, but first upgrade the kernel and make room on boot partition by manually deleting an older kernel version

Install software updates (and update kernel)

Setup up Docker apt repository

Wonder why docker-ce package is not found

Realise that Docker only supports 64 bit Linux builds, and my laptop is 32-bit

Try again with Virtualbox

Download the net installer for Ubuntu Bionic Beaver 18.0.4.1 (2 minute download)

Make a new VirtualBox image

Wait indefinitive time for VirtualBox to create a new process for the virtual machine

Realise that VirtualBox gets stuck, find solutions:

https://www.virtualbox.org/ticket/13669

https://superuser.com/questions/623989/virtual-box-stuck-at-starting-virtual-machine-0

The virtual machine's settings can not be changed for some reason

Update VirtualBox.app only to find that builtin update system fails on SSL certificate failure

Download new version of VirtualBox.app (5 minute download)

Install the new VirtualBox version

Launch the VirtualBox application, and get critical error on first launch (see screenshot):

Search for solutions to fix this

Try the solution found on StackOverflow without success

Run the uninstaller tool for the older VirtualBox version 4.3.14 (or so), that I still luckily had on my machine

Reinstall the newer VirtualBox

Now the VirtualBox application works, but complains on missing USB 2.0 support, so install newer VirtualBox additions (I wisely downloaded them also earlier)

Install 64-bit Ubuntu from the net install iso image downloaded earlier (start time: 18.18, end time: 18.38, total: 20 minutes)

Install Git and command line Emacs

Install Ubuntu Desktop because the shared clipboard does not work on command line only (start time 18.42, about 2 gigabytes download, that means a coffee break! end time: 19.00, total: 18 minutes)

Reboot and find out that Ubuntu gets stuck on boot on the second dot

Boot into recovery mode and find solutions. Hey, I am stroke, man, have you ever been stuck? Hilarious! :-D

Run sudo fsck -f / and sudo dpkg --configure -a using the CLI menu

Reboot and voilá – the machine boots into a login screen!

Install Docker-CE (start time 19.20, end time 19.25, 5 minutes)

Detour on setting the development environment

Setup Zsh (using my Zsh configuration, start time 19.33, end time 19.38, total 5 minutes)

Setup Emacs and update my configuration to work automatically (start time 19.38, end time 20.12, total time 34 minutes)

Setup GPG key for signing git commits (about ten minutes)

Commit changes to my Emacs config

Write the Dockerfile

start time: 20.45

Write the Dockerfile based on getting started example (15 minutes because on the first time I quit Emacs without saving the file and did not have the page open for copy and paste...)

[peterhil]

Tide of change

Originally written on 13th of October 2010. This is a time of high tide. If you ask any web developer right now where the web is going, she will tell you that great things are happening. But I doubt she will fully realise just how big the change will be. It will be like a tsunami that will revolutionise the way that people use and make information. The change happening now is as much sociological as it is technical. And both aspects fuel each other. It has the potential to transform our society. People like me and you just have to ignite it. The web has changed the world, but you haven’t seen anything yet. Welcome to the information age. ## Making a mashup of the mess. Mashup is the new colloquial buzzword for searching, aggregating, mining, distilling, representing and publishing information. The cleaner and more semantic markup of HTML 5 makes it possible to weed out the relevant content out of regular web pages and develop new kind of tools on the web to gather useful, reliable, interesting and relevant information from the mesh of the web. ## One markup down. Old is new again. I will start by some notes on the techniques making the change happen. HTML 5 is finally providing some sense in marking up content after ten years of hassle and confusion with XHTML. Not to mention the troubles with IE. Finally we can: - place video in a standardised format on the web page using a simple, clean and sensible syntax - use vector graphics without hacks or crashing browsers - make animations and transitions on web pages ...without Flash With CSS3 we can rotate and transform pieces of content and use the fonts we desire. The new Mozilla 4 even allows to use video in non-rectangular frame and provides hardware accelerated WebGL to show videos and other content as 3D textures just like Quartz and Compiz do. It seems to me like once again web technologies are on par with desktop technologies. I can do all the same things on the web as I can do on the desktop. Thrilling new possibilities are continuously unfolding for web developers. ## But will this be a good thriller? Many have combated with brave ninja hackery to compensate and find a way around the quirks of Internet Explorer. That way is paved with countless wasted hours covered with sweat, tears and frustration. But what are the fruits of this drudgery? They are only surface appearances – definitely not what the web should be. To make sure all this does not happen again, the real solution is to gracefully downgrade functionality and experience for which IE9 and older versions fail to meet the standards or fail to behave nicely like other browsers do. I hope these times are coming to an end, but I fear they may not – for Microsoft does not get the web. But they unwillingly made something good. ## Tear down the nodes to code this mark up. This is a story about the ugly duck that turned out to be a swan. When you hear Lisp, just pretend nothing special happened and keep reading. The story goes like this: Brendan Eich, the developer of the language was hired to Netscape to make a HyperCard like scripting language that works on their browser. He wanted to make the language be like Scheme – a Lisp dialect with more emphasis to functional programming – and aggregated in some bright ideas like prototyping from a language called Self. That name of that language was originally Mocha, which then was renamed to LiveScript. Now, we know the language as JavaScript. And now there are some really amazing Javascript libraries for AJAX, graphics and for databases. There are people making JavaScript run on the server using Google’s V8, and they can write JS code that is exactly the same as they use on a browser. There are libraries and compilers to mix and match different languages on the browser and on the server. You can run Python on the browser or compile Lisp into Javascript on both. This is definitely an exciting time to be a developer. ## S-expressions and JSON This may be shocking, but JavaScript is Lisp in disguise. It is just well camouflaged under misleading name and ugly C like syntax. That is the reason it has become ubiquitous and Lisp have not. It has also been said that XML markup is like crappy version of Lisp’s S-Expressions, which are just linked lists marked up by parenthesis where items are separated by spaces. Out of them you can build many kinds of readily serialisable data structures – trees for example. AJAX is about manipulating the browser’s tree like model of the HTML document using data fetched by asynchronous HTTP requests to and from a web server transmitted as XML or JSON snippets. Originally the X in AJAX meant XML, which is notoriously very hard to parse correctly and effectively. The reasons for XML being hard to parse are manifold, but essentially come down to tracking escapes, comments, quotes and entities in nested tags. But JavaScript has JSON which is almost as easy to handle programmatically as parsing Lisp’s S-Expressions. This together with HTML 5 local storage and other techniques means it is now easy to implement responsive web services and applications using any language or combination of languages you choose, provided that the languages have a JSON library. ## But where does the data come and go? SQL is so yesterday. Object Databases colloquially known as NoSQL and Map-Reduce querying algorithms are often much better suited to web applications than the relational model. ## All that is very touching, but what’s next? The interfaces are changing too – the WIMP (“Windows, icon, menu, pointing device”) interface paradigm of human-computer interaction was invented in 1968 by Douglas Engelbart and his research team. After that, not much have principally changed in how people use computers, until very recently. Although multitouch interfaces are an old invention, they haven’t seen wide spread usage until everybody and their aunt had an iPhone or an Android device. Sure, there have been light pens, drawing tablets and touch screens, but they used only one point and maybe pressure for input. And now there is also iPad, Microsoft Surface, Reactable, Novation Launchpad, Yamaha Tenori-on and all other interesting new musical devices. Which brings me to OpenSound Control, which is a modern version of Midi. Essentially just an interface that enables any network enabled device to provide or receive control data from any other device. ## Final note Addition on 9th of October 2018 The changes that touch devices are imposing on UIs are already visible on Google’s Material Design UI, although I am not sure if the UI for Chrome Bookmark manager is going into good direction. That is why I have been actively developing my own bookmark management application as a Chrome browser extension for the past four days. I have the ‘add bookmark’ popup almost ready now, and there you can select the parent folder (or category) of a bookmark by filtering existing folders. I have also written a prototype of this earlier using Ruby on Rails and the Neo4j graph database, as I eventually want to be able to attach the bookmarks into multiple categories. I can not understand why most technical things are arranged into rectangular grids (SQL, UI tables etc.), and why networks and graphs are so underused in implementing more interesting and natural user interfaces.

[peterhil]