Small Learnings @discretestates - Tumblr Blog

In the last couple of days, we officially admitted the first two students to our new Masters in Machine Learning program!

#machinelearning #datascience #education #academia

I'm curious if NiceGUI could be useful for developing easy-to-use apps for our biologist collaborators. I'm not a front-end developer and definitely not a web developer. NiceGUI seems like it be more like writing a JavaFx or Swing GUI than a web app.

Just learned about Optaplanner. This seems like a super cool library for bringing optimization to a larger audience by making it easy to use.

#optimization #java #scala

The micro editor seems to be a great replacement for emacs. Sits somewhere between emacs and nano in terms of functionality.

Got Apache Spark reading data from PostgreSQL. Pretty neat!

Officially on break for the next two weeks. I hope everyone has great holidays!

Re-learning everything I forgot about Apache Hadoop and Apache Spark. Spark has definitely improved since I last used it seven years ago.

Reading about massive gains in high school graduation rates in Alabama and West Virginia.

"In particular, the two states focused on what Balfanz called an early warning system, tracking behavior, attendance and grades in the ninth grade, a critical point at which many future dropouts fall through the cracks in the transition from middle school to high school."

"This data-driven approach allowed teachers to target specific students and figure out what was keeping them out of class or causing them to fail, whether it be work, family, bullies or social isolation. It was “nothing dramatic,” Balfanz said. “Just lots of problem-solving and small efforts that help students stay on track.”"

We see an analogous pattern in the first-year of college. Very excited for and proud of the work that MSOE is doing from top to bottom to ensure success of our students.

Recent class prep efforts have been focused on structured logging for services, log collection, and log storage using object stores

#ClassPrep #MLProductionSystems

I have the D programming language running on my Apple Silicon machine. The LLVM-based D Compiler supports macOS/arm64. Note that I have to build everything in release mode ("-b release") to avoid a linking error. I'm looking forward to playing with D by implementing VLists, hash trees, and tree lists.

VLists are an interesting combination of arrays and linked lists. Each node contains an array of elements. When space is needed, a new array is added to the front of the linked list. To achieve O(log n) random access, each new array is double the size of the previous array. This same strategy is used in scalable bloom filters.

Spent yesterday preparing lectures on statistical regression models and survival analysis for my Winter data science class. Reviewed likelihood functions and ratio tests, Wald tests, and pseudo R squared. Learned about predicting hazard functions with survival trees and SVMs.

Fixed more bugs in my rec sys algorithm implementations. Now seeing a clear difference in the three methods. Average ratings remains "okay" (67.3% AUC). kNN shows the impact of the relatively small number of predicted ratings that are generated (59.9% AUC). SVD demonstrates the best overall performance (92.6%), which is consistent with the literature.

Today’s #RecommendationSystems lesson: only return the N items per user with the highest predicted ratings. Otherwise, you’ll generate a query users x all items dense matrix. Oops.

Predicting ratings with truncated SVD gives accurate rankings but not raw scores. Trying to figure out where I should be normalizing / scaling.

Enjoying the new work setup.

“Monoclonal antibody prevents malaria infection in African adults”. Exciting work on an important problem.

Trending Blogs

Recently Viewed Blogs

Small Learnings