#data synchronisation

[peterhil]

Spellbook Browser extension

These are my latest notes about how to implement a bookmark manager browser extension. Maybe there is some interesting information for somebody thinking of implementing something similar.

The screenshots are from actual simple extension that I have developed in the past four days using a previous unfinished prototype.

It shows a list of filtered bookmark categories when you start to type in a category name, but there is still some usability problems.

This is implemented using Vue.js without Redux or Vuex, and uses Semantic UI as a CSS framework. I use RxJS version 6, even though it is a huge library even when trying to slim the build down using webpack.

I switched to Semantic UI from Element UI because the latter wrote styles with Javascript and used negative margins, and I see both as huge warning signs for not using that CSS framework!

The extension will be a quite simple one for now – it will not use a graph database or have syncing features.

I just want to have and use something better than the current Material UI bookmark manager that came with Google Chrome version 69, because UI wise it is not good for handling the 15.000 bookmarks and over thousand categories of bookmarks that I have.

Implementation plan for the bookmark manager

3.7.2018

In order to get something releasable done, this is the plan:

Use Redux and implement browser extension using some JS tech – if this can be used, then the bookmarks can be easily stored in IndexedDB.

JS tech

Implement a browser extension using:

Vue.js (React may make more sense with a reactive library)

Redux (is it really needed?)

Most.js, RxJS v6, Kefir, Highland.js or Bacon.js? (see perf comparison by Most)

maybe Ocaml/Reason, ClojureScript or at least ES6+?

Utility libraries:

Ramda

Nanoid

So options are:

React, Redux and Most.js with redux-most (See article 2 below)

Vue.js, Redux and Most.js with redux-vue or Vuedeux and redux-most?

Redux-observable, Redux, Vue/React ...

Redux and/or RxJS

Browser extensions operate in a very asynchronous manner, so maybe using RxJS would be a good idea?

Google Search: Redux and RxJS

Async actions in Redux with RxJS and Redux Observable

How to make your React app fully functional, fully reactive, and able to handle all those crazy… React + Redux + React-observable/react-cycles (comparable to redux-most!)

Example code:

https://dev.to/andrejnaumovski/async-actions-in-redux-with-rxjs-and--redux-observable-efg

Article https://ivanjov.com/reactive-redux-state-with-rxjs/ and Github https://github.com/IvanJov/react-rxjs-github-search

Alternative languages and tech

Vue vs React

comment on How and why we moved to Vue.js

Against using RxJS

RxJS: An object lesson in terrible good software | Isotoma: Our blog

On the other hand, see this comment on Why isn't RxJS more popular? Are there bad parts about it that's slowing adoption? : javascript ”One of the big problems with RxJS has actually been addressed in version 5, which is that it's really hard to debug. Version 4 uses a trampoline architecture, which essentially knocks over the call stack, and renders them pretty useless. You're also usually super deep into library code, which is not where you want to be. Version 5 uses a different approach, and is supposed to make the debugging situation much better.”

Why isn't RxJS more popular? Are there bad parts about it that's slowing adoption? : javascript

Unredux

This repo features a number of realistic apps built without (frontend) frameworks. We use a dataflow similar to CycleJS built on Kefir and React.

We weren't happy with clunky imperative architectures Redux, VueJS, MobX, and GraphQL apps end up with, so we decided to codify our own approach. Like CycleJS, it's 100% reactive (declarative in time!). Unlike CycleJS, it doesn't hide side-effects in drivers. See docs for more details about our motivations and design decisions.

Unredux

Unredux post edit example

ClojureScript (and core.sync like async/await in ES6)

Why I moved from RxJS to ClojureScript

Promises and Async/await in ES6+

Even with async/await, raw promises are still key to writing optimal concurrent javascript

Storage

Storage consists of:

Bookmarks

Categories

Category structure

Bookmark categories

Redux

Think and experiment about caching/indexes, read Using the Redux Store Like a Database – Hacker Noon!

Immediately after getting first version working, use large dataset to find out performance problems early!

IndexedDB

Mozilla

Google documentation

Chrome storage sync (asynchronous bulk read and write) has some severe limitations, however an extension can have unlimitedStorage permission for local storage (synchronous). Maybe another way to sync is needed?

IndexedDB with Redux

redux indexeddb - Startpage Web Search

prateekbh/redux-storage-engine-indexed-db: indexedDb engine for redux-storage

react-stack/redux-storage: Persistence layer for redux with flexible backends

Using the Redux Store Like a Database – Hacker Noon

reduxjs/reselect: Selector library for Redux

Normalizer | Redux without Profanity

javascript - How to integrate Redux with very large data-sets and IndexedDB - Stack Overflow

Best Practices for Using IndexedDB  |  Web Fundamentals  |  Google Developers

IndexedDB with promises and async/await – Filip Vitas – Medium

Offline-First Web Application Using Redux and PouchDB

Open questions

Validation

Validation on Redux store (and when syncing)?

Caching bookmark categories

into Redux store?

using binary trees?

Data synchronisation

How to synchronise bookmarks between different browsers?

GUN database could be one option, even though it’s graph API is not very developer friendly. But using that would spare me writing a CRDT algorithm.

Conflict free replicated data types (CRDTs)

Practical Demystification of CRDTs - Speaker Deck orbitdb/crdts: A library of Conflict-Free Replicated Data Types for JavaScript

dominictarr/crdt: Commutative Replicated Data Types for easy collaborative/distributed systems.

orbitdb/orbit-db: Peer-to-Peer Databases for the Decentralized Web

pfrazee/crdt_notes

Conflict-free Replicated Data Types

A Conflict-Free Replicated JSON Datatype | Hacker News

1201.1784.pdf

Conflict-free Replicated Data Types

Home - Postman Documentation

PsychoLlama/graph-crdt: Commutative graphs made for real-time, offline-tolerant replication

Automerge: JSON-like data structure for building collaborative apps | Hacker News

[paulbunting8472]

Sync SQL data in large scale using Azure SQL Data Sync

Sync SQL data in large scale using Azure SQL Data Sync

Azure SQL Data Sync allows users to synchronize data between Azure SQL Databases and SQL Server databases in one-direction or bi-direction. This feature was first introduced in 2012. By that time, people didn’t host a lot of large databases in Azure. Some size limitations were applied when we built the data sync service, including up to 30 databases (five on-premises SQL Server databases) in a…

View On WordPress

[simondtechnology]

5 Ways to Be More Efficient at Work

Are online tools in the workplace a distraction or an advantage? The truth is, if you find the right solutions for your company’s needs, there are a lot of ways that you can use technology as a powerful path to efficiency. Let’s take a look at some of the ways we can use to maximize our use of time at work.

Reducing the clutter

In an age when technology is omnipresent in the workplace, it’s easy for what is designed to be a helpful tool to become a source of distraction, clutter, and procrastination. Sometimes, things like Facebook or other social networking sites, while serving as viable options for publicity and keeping in touch with clients, can eat away at our workplace productivity. If you use technology to your advantage, however, using tools like file synchronisation or project management platforms can work powerfully in your favour in pursuit of streamlining how you and your co-workers reach your goals.

On a basic level, letting your office go as paperless as possible is an immense boon to efficiency in itself, in addition to reducing expenditures and respecting the environment. There is software or cloud technology for almost every business need, whether it is tracking your finances and invoicing or reaching your clients.

Online scheduling and project management

One of the best things you can do to maximize your efficiency at work is to make communication between employees as easily as possible. Services like Skype, Google +, and a number of cloud services provide the opportunity to easily set up meetings and conferences between co-workers regardless of geographic restrictions or platform. Additionally, there are lots of options for online scheduling, even as simple as Google Calendar, which synchronizes well across devices. Additionally, there are a wide selection of project management web platforms, like Basecamp or Wrike, that allow you a great deal of control over task assignment, tracking, and the ability to visualise long-term data and pinpoint areas where efficiency can be improved.

These solutions also benefit from cross-platform integration, a key element of modern workplace bang-for-your-buck. Intelligent mobile use is an important ally for any company. The number of apps that can be effectively implemented at work is growing every day, and includes everything from simple task management to clever employee reward systems and motivation.

Cloud computing and data synchronisation

One of the very best techniques for streamlining your work efficiency is investing in a cloud computing solution. There are myriad benefits to this technology for a large variety of different workplaces. Cloud computing is simply a way to store your company data on an off-site “cloud” server, allowing you to access, share and synchronise your files from anywhere with an internet connection, on any equipped mobile device or computer.

Users of cloud computing solutions are also able to automatize a number of network tasks, such as security concerns or backing up important files, freeing up valuable IT time. This technology is more affordable than ever, and cloud providers often provide scalable plans to fit businesses of any size and shape. It gives you an unprecedented flexibility when dealing with your files, as you are able to move them across platforms, communicate easily with co-workers, and work on-the-go, all solid benefits for your workplace efficiency. Make technology your ally, not your enemy, and cut the clutter.