Discover Top Posts Tagged with #parameter set using values from existing query

My Browser Extensions & Userscripts

The browsers I’ve used are Opera (& Opera GX), Firefox, Brave, Google Chrome, Ungoogled Chromium, Microsoft Edge, Vivaldi, and Waterfox. These are the extensions I use on them. For any extensions that have to do with downloading videos, see my Video Downloaders post that includes not just extensions, but also download managers, websites, and more.

Notes for Opera, Brave, Vivaldi, Waterfox, and Ungoogled Chromium:

Opera has it’s own add-on store along with the capability to download Chrome extensions. To download Chrome extensions all you have to do is download this extension.

Brave and Vivaldi don’t have their own add-on stores, but you can download any extensions in the Chrome store like you would in Chrome without having to do anything beforehand.

Waterfox is a fork of FIrefox that works with legacy add-ons and most current Firefox add-ons in the Firefox store. To get legacy add-ons, download the classic archive XPI from GitHub.The legacy add-ons that I use that aren’t usable in Firefox are Simple Add-On Manager (it lets you enable/disable extensions, themes, and plug-ins easily), Chrome Store Foxifed (converts Chrome CRX extensions into Firefox XPI extensions), and BarTab Plus (automatically unloads inactive tabs). For any current Firefox extensions that say they aren’t compatible, you can just got to see version history of the the add-ons in the Firefox store and download an latest older version that is compatible and if none are compatible, check the classic add-ons.

For ungoogled chromium, you have to change chrome://flags/#extension-mime-request-handling to Always Prompt for Install and get the .crx files from sites like crx4chrome or crxextractor.

AdBlocker (ESSENTIAL TO HAVE ONE... Also don’t use AdBlock or AdBlock Plus... Please choose EITHER AdGuard or uBlock Origin)

AdGuard (Available for Chrome/Brave/Vivaldi/Yandex, Opera, Firefox, Edge, & Safari on computer + they also have a free app for Android & iOS [Safari on iPhone & iPad])

AdGuard is my favorite adblocking browser extension. I used to use uBlock Origin + Nano Defender, which are also excellent, but then I discovered AdGuard, which I liked even better and uninstalled those since you shouldn’t use more than one adblocker in your browser. The reason I love AdGuard is because it works while still enabled on sites where you would have to disable uBlock Origin to get the site to work (ie. stream.nbcsports.com, youku.com).

The default settings of AdGuard are already great (just AdGuard base filter enabled), but you can enable even more filter lists and settings for the best browser experience. The ones I enable to block the most ads/annoyances/get the best privacy, while also not breaking sites I use are: AdGuard Base Filter, Peter Lowe’s List, AdGuard Tracking Protection Filter, Easy Privacy, AdGuard Social Media Filter, AdGuard Annoyances Filter, Adblock Warning Removal List, Malware Domains, Spam404, NoCoin Filter List, and Filter unblocking search ads and self-promotion. I also have Phishing & Malware Protection and Stealth Mode (Self Destructing 3rd party cookies with 2880 lifetime, Hide Referrer from third-parties, Hide your search queries, Send Do-Not-Track header, Remove Tracking Parameters) enabled.

Social Media/Enhancers

Good Twitter (Chrome/Brave/Opera/Vivaldi), Firefox/Waterfox)

Light extensions that change your user agent to Mozilla/5.0 (Windows NT 9.0; WOW64; Trident/7.0; rv:11.0) on only twitter.com to give you the previous twitter desktop layout back.

If you have Firefox or Waterfox, you can also change your twitter back without an extension by going to about:config, find the general.useragent.site_specific_overrides setting, right click, choose new string, enter general.useragent.override.twitter.com as the preference name, and Mozilla/5.0 (Windows NT 9.0; WOW64; Trident/7.0; rv:11.0) as the string value.

New XKit (Chrome/Brave/Opera/Vivaldi, Firefox/Waterfox, Safari)

Note: For Firefox there is a version in the Firefox add-ons store, but it is only version 7.8.2 instead of the latest 7.9.0, so I’m linking the github xpi instead. You can just click on the xpi and Firefox should display continue to extension from github, click continue, and then the normal add add-on pop that you get for any extensions should show up and just click add/confirm your installation. If that doesn’t work, you can save the xpi to your computer, go to your Firefox add-ons page, click “install add-on from file”, choose the xpi, and then the normal add-on pop up will appear and you can click add/confirm your installation.

An amazing extension that makes using Tumblr 1000x better and allows you to add Tumblr enhancement extensions to tumblr.com. XInbox, One-Click Postage, One-Click Reply, Tweaks, and XCloud are already installed by default. The other extensions I have installed are Go-To-Dash, Don’t Stretch Photosets, Timestamps, Soft Refresh, Search Likes, Mutual Checker, Find Inactives, Outbox, Post Archiver, Post Limit Checker, Show Picture Size, Activity+, Anti-Capitalism, Drafts+, Mass+, Read More Now, Quick Tags, Reply Viewer, TagViewer, and View on Dash.

Chrome IG Story (Opera/Ungoogled Chromium)

Note: This was originally an extension for Chrome, but got removed from the Chrome web store. I already had it installed from the web store on Chrome and it continued to work even after it was removed. However, you can’t install it for Chrome or Brave or Vivaldi anymore even with the CRX file because it will give you a “CRX Header Invalid” error. From the site I linked, you can still install it for Opera though by simply clicking "Download from Google CDN” or “Download from Crx4Chrome.” You can also install it on Ungoogled Chromium, if you have chrome://flags/#extension-mime-request-handling set to Always Prompt for Install like I mentioned in the note at the beginning of my post.

An extension that lets you view, download, and get direct URLs for instagram stories and live videos on the web version.

Reddit Enhancement Suite (Chrome/Brave/Opera/Vivaldi, Firefox, Waterfox (use version 5.12.8), Edge)

Reddit Enhancement Suite is a suite of tools to enhance your Reddit browsing experience. It includes features such as: - Never Ending Reddit - never click "next page" again! - Inline Image Viewer - adds buttons to view images without leaving the page, including support for imgur albums and more - Keyboard Navigation - browse reddit more efficiently than ever before with keyboard shortcuts for most functionality - Uppers and Downers - see the hidden vote totals that Reddit provides behind the scenes - Account Switcher - switch accounts easily and quickly - User Tagger - keep track of users you run across frequently, complete with color coded tags, etc.

An extension that makes your reddit experience way better. It works best when you use the old reddit (you can opt out of new reddit design in your account or you can just go to old.reddit.com) aka the superior reddit.

Enhancer for YouTube (Chrome/Brave/Opera/Vivaldi, Firefox/Waterfox, Edge)

Tons of features to improve your user experience on YouTube™:

✔ Control volume level and playback speed with the mouse wheel ✔ Remove ads from videos (automatically or on-demand) ✔ Whitelist channels to not automatically remove their ads ✔ Remove annotations (automatically or on-demand) ✔ Automatically play videos in 4K, HD, or any other preferred format ✔ Loop videos (in part or in whole) ✔ Use custom themes ✔ Use a larger video player ✔ Pin the video player ✔ Execute your own actions using JavaScript ✔ Disable next video autoplay

Social Fixer (Chrome/Brave/Vivaldi, Opera, Firefox/Waterfox, Safari, Userscript)

Social Fixer for Facebook fixes annoyances, adds features, and enhances existing functionality to make FB more fun and efficient. Filter the news feed, hide sponsored posts and political posts, hide parts of the page you don't want to see, and more!

Thread Reader (Chrome/Brave/Opera/Vivaldi, Firefox/Waterfox)

This adds an “Unroll in Thread Reader” option under the arrow with the more options in a tweet, which directly takes you the unrolled thread, making it easier to read twitter threads without having to tweet at the thread reader bot or manually enter URLs on their website.

Userscripts

You can use userscripts by downloading the userscript manager extensions: Tampermonkey (Chrome/Brave/Vivaldi, Opera, Firefox/Waterfox, Edge, Safari) or Violentmonkey (Chrome/Brave/Opera/Vivaldi, Firefox/Waterfox).

On Chromium browsers, Waterfox, and Edge I use Tampermonkey because the AdGuard Popup Blocker only works with Tampermonkey for me and VK Media Downloader still works with it. On Firefox, I use Violentmonkey because VK Media Downloader stopped working for me with Tampermonkey on Firefox on both my Windows 10 and Linux computer. AdGuard Popup Blocker doesn’t work with Violentmonkey for me (or on my Windows 10 computer even with Tampermonkey, though AdGuard Popup Blocker does work with Firefox/Tampermonkey for me on Linux), so I also use Popup Blocker (strict) (Chrome/Brave/Vivaldi, Opera, Firefox, Edge) on Firefox.

The userscripts I use are:

AntiAdware- Remove forced download accelerators, managers, and adware on supported websites

AdGuard Popup Blocker- Blocks popups on all websites

Resize Image On “Open Image In New Tab”- Super useful userscript that automatically opens images in their original/largest size (especially useful for tumblr images from text posts that get cut down to like 500 width, but with this they go back to their original size like 540 or 1280)

AdsByPasser- Skips countdown ads or continue pages or shortened links

Local YouTube Downloader- Shows all direct YouTube googlevideo URLs under each YouTube video

VK Media Downloader- Adds a download button to VK videos and allows you to view or download direct VK video URLs

KissAnime Anti-Adblock Blocker- Removes the cruft, obtrusive advertising and their Anti-Adblock nuisance screen

KissAnime Complete captcha removal- After installing this script you'll probably forget about captchas in Kissanime for good, because it jumps directly to the video without passing by the captcha's page. The only catch is that you'll be using Rapidvideo server as long as this script is activated. (In case you choose a different server you'll have to manually answer the captcha)

#browser extensions #technology #finally finished this post

Something awesome - web research

In order to better understand web exploitation as a concept, I need to first gain a better understanding of how networks are structured, and how information is sent over the internet.

You can read the notes I’ve compiled below:

#somethingawesome

Serving article comments using neural nets and reinforcement learning

Yahoo properties such as Yahoo Finance, Yahoo News, and Yahoo Sports allow users to comment on the articles, similar to many other apps and websites. To support this we needed a system that can add, find, count and serve comments at scale in real time. Not all comments are equally as interesting or relevant though, and some articles can have hundreds of thousands of comments, so a good commenting system must also choose the right comments among these to show to users viewing the article. To accomplish this, the system must observe what users are doing and learn how to pick comments that are interesting.

In this blog post, we’ll explain how we’re solving this problem for Yahoo properties by using Vespa - the open source big data serving engine. We’ll start with the basics and then show how comment selection using a neural net and reinforcement learning has been implemented.

Real-time comment serving

As mentioned, we need a system that can add, find, count, and serve comments at scale in real time. Vespa allows us to do this easily by storing each comment as a separate document, containing the ID of the article commented upon, the ID of the user commenting, various comment metadata, and the comment text itself. Vespa then allows us to issue queries to quickly retrieve the comments on a given article for display, or to show a comment count next to the article:

Ranking comments

In addition, we can show all the articles of a given user and similar less-used operations.

We store about a billion comments at any time, serve about 12.000 queries per second, and about twice as many writes (new comments + comment metadata updates). Average latency for queries is about 4 ms, and write latency roughly 1 ms. Nodes are organized in two tiers as a single Vespa application: A single stateless cluster handling incoming queries and writes, and a content cluster storing the comments, maintaining indexes and executing the distributed part of queries in parallel. In total, we use 32 stateless and 96 stateful nodes spread over 5 regional data centers. Data is automatically sharded by Vespa in each datacenter, in 6-12 shards depending on the traffic patterns of that region.

Some articles have a very large number of comments - up to hundreds of thousands are not uncommon, and no user is going to read all of them. Therefore we need to pick the best comments to show each time someone views an article. To do this, we let Vespa find all the comments for the article, compute a score for each, and pick the comments with the best scores to show to the user. This process is called ranking. By configuring the function to compute for each comment as a ranking expression in Vespa, the engine will compute it locally on each data partition in parallel during query execution. This allows us to execute these queries with low latency and ensures that we can handle more comments by adding more content nodes, without causing an increase in latency.

The input to the ranking function is features which are typically stored in the comment or sent with the query. Comments have various features indicating how users interacted with the comment, as well as features computed from the comment content itself. In addition, we keep track of the reputation of each comment author as a feature.

User actions are sent as update operations to Vespa as they are performed. The information about authors is also continuously changing, but since each author can write many comments it would be wasteful to have to update each article everytime we have new information about the author. Instead, we store the author information in a separate document type - one document per author and use a document reference in Vespa to import that author feature into each comment. This allows us to update author information once and have it automatically take effect for all comments by that author.

With these features, we can configure a mathematical function as a ranking expression which computes the rank score or each comment to produce a ranked list of the top comments, like the following:

Using a neural net and reinforcement learning

We used to rank comments using a handwritten ranking expression with hardcoded weighting of the features. This is a good way to get started but obviously not optimal. To improve it we need to decide on a measurable target and use machine learning to optimize towards it.

The ultimate goal is for users to find the comments interesting. This can not be measured directly, but luckily we can define a good proxy for interest based on signals such as dwell time (the amount of time the users spend on the comments of an article) and user actions (whether users reply to comments, provide upvotes and downvotes, etc). We know that we want user interest to go up on average, but we don’t know what the correct value of this measure of interest might be for any given list of comments. Therefore it’s hard to create a training set of interest signals for articles (supervised learning), so we chose to use reinforcement learning instead: Let the system make small changes to the live machine-learned model iteratively, observe the effect on the signal we use as a proxy for user interest, and use this to converge on a model that increases it.

The model chosen is a neural net with multiple hidden layers, roughly illustrated as follows:

The advantage of using a neural net compared to a simple function such as linear regression is that we can capture non-linear relationships in the feature data without having to guess which relationship exists and hand-write functions to capture them (feature engineering).

To explore the space of possible rankings, we implement a sampling algorithm in a Searcher to perturb the ranking of comments returned from each query. We log the ranking information and our user interest signals such as dwell time to our Hadoop grid where they are joined. This generates a training set each hour which we use to retrain the model using TensorFlow-on-Spark, which generates a new model for the next iteration of the reinforcement learning.

To implement this on Vespa, we configure the neural net as the ranking function for comments. This was done as a manually written ranking function over tensors in a rank profile:

rank-profile neuralNet {

function get_model_weights(field) {

expression: if(query(field) == 0, constant(field), query(field))

}

function layer_0() { # returns tensor(hidden[9])

expression: elu(xw_plus_b(nn_input,

get_model_weights(W_0),

get_model_weights(b_0),

x))

}

function layer_1() { # returns tensor(out[9])

expression: elu(xw_plus_b(layer_0,

get_model_weights(W_1),

get_model_weights(b_1),

hidden))

}

function layer_out() { # xw_plus_b returns tensor(out[1]), so sum converts to double

expression: sum(xw_plus_b(layer_1,

get_model_weights(W_out),

get_model_weights(b_out),

out))

}

first-phase {

expression: freshnessRank

}

second-phase {

expression: layer_out

rerank-count: 2000

}

More recently Vespa added support for deploying TensorFlow SavedModels directly, which would also be a good option since the training happens in TensorFlow.

Neural nets have a pair of weight and bias tensors for each layer, which is what we want our training process to optimize. The simplest way to include the weights and biases in the model is to add them as constant tensors to the application package. However, to do reinforcement learning we need to be able to update them frequently. We could achieve this by redeploying the application package frequently, as Vespa allows this to be done without restarts or disruption to ongoing queries. However, it is still a somewhat heavy-weight process, so we chose another approach: Store the neural net parameters as tensors in a separate document type, and create a Searcher component which looks up this document on each incoming query, and adds the parameter tensors to it before it’s passed to the content nodes for evaluation.

Here is the full code needed to accomplish this:

import com.yahoo.document.Document;

import com.yahoo.document.DocumentId;

import com.yahoo.document.Field;

import com.yahoo.document.datatypes.FieldValue;

import com.yahoo.document.datatypes.TensorFieldValue;

import com.yahoo.documentapi.DocumentAccess;

import com.yahoo.documentapi.SyncParameters;

import com.yahoo.documentapi.SyncSession;

import com.yahoo.search.Query;

import com.yahoo.search.Result;

import com.yahoo.search.Searcher;

import com.yahoo.search.searchchain.Execution;

import com.yahoo.tensor.Tensor;

import java.util.Map;

public class LoadRankingmodelSearcher extends Searcher {

private static final String VESPA_DOCUMENTID_FORMAT = "id:canvass_search:rankingmodel::%s";

// https://docs.vespa.ai/documentation/ranking.html#using-query-variables:

private static final String QUERY_FEATURE_FORMAT = "query(%s)";

/** To fetch model documents from Vespa index */

private final SyncSession fetchDocumentSession;

public LoadRankingmodelSearcher() {

this.fetchDocumentSession = DocumentAccess.createDefault().createSyncSession(new SyncParameters.Builder().build());

}

@Override

public Result search(Query query, Execution execution) {

// fetch model document from Vespa

String documentId = String.format(VESPA_DOCUMENTID_FORMAT, query.getRanking().getProfile());

Document modelDoc = fetchDocumentSession.get(new DocumentId(documentId));

// Add it to the query

if (modelDoc != null) {

modelDoc.iterator().forEachRemaining((Map.Entry<Field, FieldValue> e) ->

addTensorFromDocumentToQuery(e.getKey().getName(), e.getValue(), query)

);

}

return execution.search(query);

}

private static void addTensorFromDocumentToQuery(String field, FieldValue value, Query query) {

if (value instanceof TensorFieldValue) {

Tensor tensor = ((TensorFieldValue) value).getTensor().get();

query.getRanking().getFeatures().put(String.format(QUERY_FEATURE_FORMAT, field), tensor);

}

The model weight document definition is added to the same content cluster as the comment documents and simply contains attribute fields for each weight and bias tensor of the neural net:

document rankingmodel {

field modelTimestamp type long { … }

field W_0 type tensor(x[9],hidden[9]){ … }

field b_0 type tensor(hidden[9]){ … }

field W_1 type tensor(hidden[9],out[9]){ … }

field b_1 type tensor(out[9]){ … }

field W_out type tensor(out[9]){ … }

field b_out type tensor(out[1]){ … }

}

Since updating documents is a lightweight operation we can now make frequent changes to the neural net to implement the reinforcement learning.

Results

Switching to the neural net model with reinforcement learning led to a 20% increase in average dwell time. The average response time when ranking with the neural net increased to about 7 ms since the neural net model is more expensive. The response time stays low because in Vespa the neural net is evaluated on all the content nodes (partitions) in parallel. We avoid the bottleneck of sending the data for each comment to be evaluated over the network and can increase parallelization indefinitely by adding more content nodes.

However, evaluating the neural net for all comments for outlier articles which have hundreds of thousands of comments would still be very costly. If you read the rank profile configuration shown above, you’ll have noticed the solution to this: We use two-phase ranking where the comments are first selected by a cheap rank function (which we term freshnessRank) and the highest scoring 2000 documents (per content node) are re-ranked using the neural net. This caps the max CPU spent on evaluating the neural net per query.

Conclusion and future work

We have shown how to implement a real comment serving and ranking system on Vespa. With reinforcement learning gaining popularity, the serving system needs to become a more integrated part of the machine learning stack, and by using Vespa and TensorFlow-on-Spark, this can be accomplished relatively easily with a standard open source technology.

We plan to expand on this work by applying it to other domains such as content recommendation, incorporating more features in a larger network, and exploring personalized comment ranking.

Acknowledgments

Thanks to Aaron Nagao, Sreekanth Ramakrishnan, Zhi Qu, Xue Wu, Kapil Thadani, Akshay Soni, Parikshit Shah, Troy Chevalier, Sreekanth Ramakrishnan, Jon Bratseth, Lester Solbakken and Håvard Pettersen for their contributions to this work.

#big data #AI #artificial intelligence #machine learning #tensorflow #java #search engine #open source

Serving article comments using reinforcement learning of a neural net

Don’t look at the comments. When you allow users to make comments on your content pages you face the problem that not all of them are worth showing — a difficult problem to solve, hence the saying. In this article I’ll show how this problem has been attacked using reinforcement learning at serving time on Yahoo content sites, using the Vespa open source platform to create a scalable production solution.

Yahoo properties such as Yahoo Finance, News and Sports allow users to comment on the articles, similar to many other apps and websites. To support this the team needed a system that can add, find, count and serve comments at scale in real time. Not all comments are equally as interesting or relevant though, and some articles can have hundreds of thousands of comments, so a good commenting system must also choose the right comments among these to show to users viewing the article. To accomplish this, the system must observe what users are doing and learn how to pick comments that are interesting.

Here I’ll explain how this problem was solved for Yahoo properties by using Vespa — the open source big data serving engine. I’ll start with the basics and then show how comment selection using a neural net and reinforcement learning was implemented.

Real-time comment serving

As mentioned, the team needed a system that can add, find, count, and serve comments at scale in real time. The team chose Vespa, the open big data serving engine for this, as it supports both such basic serving as well as incorporating machine learning at serving time (which we’ll get to below). By storing each comment as a separate document in Vespa, containing the ID of the article commented upon, the ID of the user commenting, various comment metadata, and the comment text itself, the team could issue queries to quickly retrieve the comments on a given article for display, or to show a comment count next to the article:

In addition, this document structure allowed less-used operations such as showing all the articles of a given user and similar.

The Vespa instance used at Yahoo for this store about a billion comments at any time, serve about 12.000 queries per second, and about twice as many writes (new comments + comment metadata updates). Average latency for queries is about 4 ms, and write latency roughly 1 ms. Nodes are organized in two tiers as a single Vespa application: A single stateless cluster handling incoming queries and writes, and a content cluster storing the comments, maintaining indexes and executing the distributed part of queries in parallel. In total, 32 stateless and 96 stateful nodes are spread over 5 regional data centers. Data is automatically sharded by Vespa in each datacenter, in 6–12 shards depending on the traffic patterns of that region.

Ranking comments

Some articles on Yahoo pages have a very large number of comments — up to hundreds of thousands are not uncommon, and no user is going to read all of them. Therefore it is necessary to pick the best comments to show each time someone views an article. Vespa does this by finding all the comments for the article, computing a score for each, and picking the comments with the best scores to show to the user. This process is called ranking. By configuring the function to compute for each comment as a ranking expression in Vespa, the engine will compute it locally on each data partition in parallel during query execution. This allows executing these queries with low latency and ensures that more comments can be handled by adding more content nodes, without causing an increase in latency.

The input to the ranking function is features which are typically stored in the document (here: a comment) or sent with the query. Comments have various features indicating how users interacted with the comment, as well as features computed from the comment content itself. In addition, the system keeps track of the reputation of each comment author as a feature.

User actions are sent as update operations to Vespa as they are performed. The information about authors is also continuously changing, but since each author can write many comments it would be wasteful to have to update each comment every time there is new information about the author. Instead, the author information is stored in a separate document type — one document per author, and a document reference in Vespa is used to import that author feature into each comment. This allows updating the author information once and have it automatically take effect for all comments by that author.

With these features, it’s possible in Vespa to configure a mathematical function as a ranking expression which computes the rank score or each comment to produce a ranked list of the top comments, like the following:

Using a neural net and reinforcement learning

The team used to rank comments with a handwritten ranking expression having hardcoded weighting of the features. This is a good way to get started but obviously not optimal. To improve it they needed to decide on a measurable target and use machine learning to optimize towards it.

The ultimate goal is for users to find the comments interesting. This can not be measured directly, but luckily we can define a good proxy for interest based on signals such as dwell time (the amount of time the users spend on the comments of an article) and user actions (whether users reply to comments, provide upvotes and downvotes, etc). The team knew they wanted user interest to go up on average, but there is no way to know what the correct value of the measure of interest might be for any single given list of comments. Therefore it’s hard to create a training set of interest signals for articles (supervised learning), so reinforcement learning was chosen instead: Let the system make small changes to the live machine-learned model iteratively, observe the effect on the signal used as a proxy for user interest, and use this to converge on a model that increases it.

The model chosen here was a neural net with multiple hidden layers, roughly illustrated as follows:

The advantage of using a neural net compared to a simple function such as linear regression is that it can capture non-linear relationships in the feature data without anyone having to guess which relationship exists and hand-write functions to capture them (feature engineering).

To explore the space of possible rankings, the team implemented a sampling algorithm in a Searcher to perturb the ranking of comments returned from each query. They logged the ranking information and user interest signals such as dwell time to their Hadoop grid where they are joined. This generates a training set each hour which is used to retrain the model using TensorFlow-on-Spark, which produces a new model for the next iteration of the reinforcement learning cycle.

To implement this on Vespa, the team configured the neural net as the ranking function for comments. This was done as a manually written ranking function over tensors in a rank profile. Here is the production configuration used:

rank-profile neuralNet { function get_model_weights(field) { expression: if(query(field) == 0, constant(field), query(field)) } function layer_0() { # returns tensor(hidden[9]) expression: elu(xw_plus_b(nn_input, get_model_weights(W_0), get_model_weights(b_0), x)) } function layer_1() { # returns tensor(out[9]) expression: elu(xw_plus_b(layer_0, get_model_weights(W_1), get_model_weights(b_1), hidden)) } # xw_plus_b returns tensor(out[1]), so sum converts to double function layer_out() { expression: sum(xw_plus_b(layer_1, get_model_weights(W_out), get_model_weights(b_out), out)) } first-phase { expression: freshnessRank } second-phase { expression: layer_out rerank-count: 2000 } }

More recently Vespa added support for deploying TensorFlow SavedModels directly (as well as similar support for tools saving in the ONNX format), which would also be a good option here since the training happens in TensorFlow.

Neural nets have a pair of weight and bias tensors for each layer, which is what the team wanted the training process to optimize. The simplest way to include the weights and biases in the model is to add them as constant tensorsto the application package. However, with reinforcement learning it is necessary to be able update these tensor parameters frequently. This could be achieved by redeploying the application package frequently, as Vespa allows that to be done without restarts or disruption to ongoing queries. However, it is still a somewhat heavy-weight process, so another approach was chosen: Store the neural net parameters as tensors in a separate document type in Vespa, and create a Searcher component which looks up this document on each incoming query, and adds the parameter tensors to it before it’s passed to the content nodes for evaluation.

Here is the full production code needed to accomplish this serving-time operation:

import com.yahoo.document.Document; import com.yahoo.document.DocumentId; import com.yahoo.document.Field; import com.yahoo.document.datatypes.FieldValue; import com.yahoo.document.datatypes.TensorFieldValue; import com.yahoo.documentapi.DocumentAccess; import com.yahoo.documentapi.SyncParameters; import com.yahoo.documentapi.SyncSession; import com.yahoo.search.Query; import com.yahoo.search.Result; import com.yahoo.search.Searcher; import com.yahoo.search.searchchain.Execution; import com.yahoo.tensor.Tensor; import java.util.Map; public class LoadRankingmodelSearcher extends Searcher { private static final String VESPA_ID_FORMAT = "id:canvass_search:rankingmodel::%s"; // https://docs.vespa.ai/documentation/ranking.html#using-query-variables: private static final String FEATURE_FORMAT = "query(%s)"; /** To fetch model documents from Vespa index */ private final SyncSession fetchDocumentSession; public LoadRankingmodelSearcher() { this.fetchDocumentSession = DocumentAccess.createDefault() .createSyncSession(new SyncParameters.Builder().build()); } @Override public Result search(Query query, Execution execution) { // Fetch model document from Vespa String id = String.format(VESPA_ID_FORMAT, query.getRanking().getProfile()); Document modelDoc = fetchDocumentSession.get(new DocumentId(id)); // Add it to the query if (modelDoc != null) { modelDoc.iterator().forEachRemaining((Map.Entry<Field, FieldValue> e) -> addTensorFromDocumentToQuery(e.getKey().getName(), e.getValue(), query) ); } return execution.search(query); } private static void addTensorFromDocumentToQuery(String field, FieldValue value, Query query) { if (value instanceof TensorFieldValue) { Tensor tensor = ((TensorFieldValue) value).getTensor().get(); query.getRanking().getFeatures().put(String.format(FEATURE_FORMAT, field), tensor); } } }

The model weight document definition is added to the same content cluster as the comment documents and simply contains attribute fields for each weight and bias tensor of the neural net (where each field below is configured with “indexing: attribute | summary”):

document rankingmodel { field modelTimestamp type long { … } field W_0 type tensor(x[9],hidden[9]) { … } field b_0 type tensor(hidden[9]) { … } field W_1 type tensor(hidden[9],out[9]) { … } field b_1 type tensor(out[9]) { … } field W_out type tensor(out[9]) { … } field b_out type tensor(out[1]) { … } }

Since updating documents is a lightweight operation it is now possible to make frequent changes to the neural net to implement the reinforcement learning process.

Results

Switching to the neural net model with reinforcement learning has already led to a 20% increase in average dwell time. The average response time when ranking with the neural net increased to about 7 ms since the neural net model is more expensive. The response time stays low because in Vespa the neural net is evaluated on all the content nodes (partitions) in parallel. This avoids the bottleneck of sending the data for each comment to be evaluated over the network and allows increasing parallelization indefinitely by adding more content nodes.

However, evaluating the neural net for all comments for outlier articles which have hundreds of thousands of comments would still be very costly. If you read the rank profile configuration shown above, you’ll have noticed the solution to this: Two-phase ranking was used where the comments are first selected by a cheap rank function (termed freshnessRank) and the highest scoring 2000 documents (per content node) are re-ranked using the neural net. This caps the max CPU spent on evaluating the neural net per query.

Conclusion and future work

In this article I have shown how to implement a real comment serving and ranking system on Vespa. With reinforcement learning gaining popularity, the serving system needs to become a more integrated part of the machine learning stack, and by using Vespa this can be accomplished relatively easily with a standard open source technology.

The team working on this plan to expand on this work by applying it to other domains such as content recommendation, incorporating more features in a larger network, and exploring personalized comment ranking.

#machine learning #recommendation systems #big data #reinforcement learning #neural networks

Scaling TensorFlow model evaluation with Vespa

In this blog post we'll explain how to use Vespa to evaluate TensorFlow models over arbitrarily many data points while keeping total latency constant. We provide benchmark data from our performance lab where we compare evaluation using TensorFlow serving with evaluating TensorFlow models in Vespa.

We recently introduced a new feature that enables direct import of TensorFlow models into Vespa for use at serving time. As mentioned in a previous blog post, our approach to support TensorFlow is to extract the computational graph and parameters of the TensorFlow model and convert it to Vespa's tensor primitives. We chose this approach over attempting to integrate our backend with the TensorFlow runtime. There were a few reasons for this. One was that we would like to support other frameworks than TensorFlow. For instance, our next target is to support ONNX. Another was that we would like to avoid the inevitable overhead of such an integration, both on performance and code maintenance. Of course, this means a lot of optimization work on our side to make this as efficient as possible, but we do believe it is a better long term solution.

Naturally, we thought it would be interesting to set up some sort of performance comparison between Vespa and TensorFlow for cases that use a machine learning ranking model.

Before we get to that however, it is worth noting that Vespa and TensorFlow serving has an important conceptual difference. With TensorFlow you are typically interested in evaluating a model for a single data point, be that an image for an image classifier, or a sentence for a semantic representation etc. The use case for Vespa is when you need to evaluate the model over many data points. Examples are finding the best document given a text, or images similar to a given image, or computing a stream of recommendations for a user.

So, let’s explore this by setting up a typical search application in Vespa. We’ve based the application in this post on the Vespa blog recommendation tutorial part 3. In this application we've trained a collaborative filtering model which computes an interest vector for each existing user (which we refer to as the user profile) and a content vector for each blog post. In collaborative filtering these vectors are commonly referred to as latent factors. The application takes a user id as the query, retrieves the corresponding user profile, and searches for the blog posts that best match the user profile. The match is computed by a simple dot-product between the latent factor vectors. This is used as the first phase ranking. We've chosen vectors of length 128.

In addition, we've trained a neural network in TensorFlow to serve as the second-phase ranking. The user vector and blog post vector are concatenated and represents the input (of size 256) to the neural network. The network is fully connected with 2 hidden layers of size 512 and 128 respectively, and the network has a single output value representing the probability that the user would like the blog post.

In the following we set up two cases we would like to compare. The first is where the imported neural network is evaluated on the content node using Vespa's native tensors. In the other we run TensorFlow directly on the stateless container node in the Vespa 2-tier architecture. In this case, the additional data required to evaluate the TensorFlow model must be passed back from the content node(s) to the container node. We use Vespa's fbench utility to stress the system under fairly heavy load.

In this first test, we set up the system on a single host. This means the container and content nodes are running on the same host. We set up fbench so it uses 64 clients in parallel to query this system as fast as possible. 1000 documents per query are evaluated in the first phase and the top 200 documents are evaluated in the second phase. In the following, latency is measured in ms at the 95th percentile and QPS is the actual query rate in queries per second:

Baseline: 19.68 ms / 3251.80 QPS

Baseline with additional data: 24.20 ms / 2644.74 QPS

Vespa ranking: 42.8 ms / 1495.02 QPS

TensorFlow batch ranking: 42.67 ms / 1499.80 QPS

TensorFlow single ranking: 103.23 ms / 619.97 QPS

Some explanation is in order. The baseline here is the first phase ranking only without returning the additional data required for full ranking. The baseline with additional data is the same but returns the data required for ranking. Vespa ranking evaluates the model on the content backend. Both TensorFlow tests evaluate the model after content has been sent to the container. The difference is that batch ranking evaluates the model in one pass by batching the 200 documents together in a larger matrix, while single evaluates the model once per document, i.e. 200 evaluations. The reason why we test this is that Vespa evaluates the model once per document to be able to evaluate during matching, so in terms of efficiency this is a fairer comparison.

We see in the numbers above for this application that Vespa ranking and TensorFlow batch ranking achieve similar performance. This means that the gains in ranking batch-wise is offset by the cost of transferring data to TensorFlow. This isn’t entirely a fair comparison however, as the model evaluation architecture of Vespa and TensorFlow differ significantly. For instance, we measure that TensorFlow has a much lower degree of cache misses. One reason is that batch-ranking necessitates a more contiguous data layout. In contrast, relevant document data can be spread out over the entire available memory on the Vespa content nodes.

Another significant reason is that Vespa currently uses double floating point precision in ranking and in tensors. In the above TensorFlow model we have used floats, resulting in half the required memory bandwidth. We are considering making the floating point precision in Vespa configurable to improve evaluation speed for cases where full precision is not necessary, such as in most machine learned models.

So we still have some work to do in optimizing our tensor evaluation pipeline, but we are pleased with our results so far. Now, the performance of the model evaluation itself is only a part of the system-wide performance. In order to rank with TensorFlow, we need to move data to the host running TensorFlow. This is not free, so let’s delve a bit deeper into this cost.

The locality of data in relation to where the ranking computation takes place is an important aspect and indeed a core design point of Vespa. If your data is too large to fit on a single machine, or you want to evaluate your model on more data points faster than is possible on a single machine, you need to split your data over multiple nodes. Let’s assume that documents are distributed randomly across all content nodes, which is a very reasonable thing to do. Now, when you need to find the globally top-N documents for a given query, you first need to find the set of candidate documents that match the query. In general, if ranking is done on some other node than where the content is, all the data required for the computation obviously needs to be transferred there. Usually, the candidate set can be large so this incurs a significant cost in network activity, particularly as the number of content nodes increase. This approach can become infeasible quite quickly.

This is why a core design aspect of Vespa is to evaluate models where the content is stored.

This is illustrated in the figure above. The problem of transferring data for ranking is compounded as the number of content nodes increase, because to find the global top-N documents, the top-K documents of each content node need to be passed to the external ranker. This means that, if we have C content nodes, we need to transfer C*K documents over the network. This runs into hard network limits as the number of documents and data size for each document increases.

Let’s see the effect of this when we change the setup of the same application to run on three content nodes and a single stateless container which runs TensorFlow. In the following graph we plot the 95th percentile latency as we increase the number of parallel requests (clients) from 1 to 30:

Here we see that with low traffic, TensorFlow and Vespa are comparable in terms of latency. When we increase the load however, the cost of transmitting the data is the driver for the increase in latency for TensorFlow, as seen in the red line in the graph. The differences between batch and single mode TensorFlow evaluation become smaller as the system as a whole becomes largely network-bound. In contrast, the Vespa application scales much better.

Now, as we increase traffic even further, will the Vespa solution likewise become network-bound? In the following graph we plot the sustained requests per second as we increase clients to 200:

Vespa ranking is unable to sustain the same amount of QPS as just transmitting the data (the blue line), which is a hint that the system has become CPU-bound on the evaluation of the model on Vespa. While Vespa can sustain around 3500 QPS, the TensorFlow solution maxes out at 350 QPS which is reached quite early as we increase traffic. As the system is unable to transmit data fast enough, the latency naturally has to increase which is the cause for the linearity in the latency graph above. At 200 clients the average latency of the TensorFlow solution is around 600 ms, while Vespa is around 60 ms.

So, the obvious key takeaway here is that from a scalability point of view it is beneficial to avoid sending data around for evaluation. That is both a key design point of Vespa, but also for why we implemented TensorFlow support in the first case. By running the models where the content is allows for better utilization of resources, but perhaps the more interesting aspect is the ability to run more complex or deeper models while still being able to scale the system.

Power BI - how to set parameter values using query option .

At @globalTechLearn, you will find learning tutorials for various tools like Microsoft Power BI. It has more focus on industry practical exa

n this video we have discussed how to set parameter values using query option. Using this we can set parameter values from existing query.

Hope you find this vidoe useful. Please like share and subscribe our channel and keep updated with all upcoming videos

Thank you 😊

#Power bi #Power query editor #Parameters #Parameter set using query option #Parameter set using values from existing query