#machine learning with python interview questions

[iamprogrammerz]

Top 5 Machine Learning Projects for Beginners ☞ http://dev.edupioneer.net/376e6365b3 #MachineLearning

[iamcodegeek]

Learn Machine Learning: Crash Course for Beginners ⭐️⭐️⭐️⭐️⭐️ ☞ http://on.edupioneer.net/11f99e830b #MachineLearning

[raomarketingpro]

Free AI Tools

Artificial Intelligence (AI) has revolutionized the way we work, learn, and create. With an ever-growing number of tools, it’s now easier than ever to integrate AI into your personal and professional life without spending a dime. Below, we’ll explore some of the best free AI tools across various categories, helping you boost productivity, enhance creativity, and automate mundane tasks.

Wanna know about free ai tools

1. Content Creation Tools

ChatGPT (OpenAI)

One of the most popular AI chatbots, ChatGPT, offers a free plan that allows users to generate ideas, write content, answer questions, and more. Its user-friendly interface makes it accessible for beginners and professionals alike.

Best For:

Writing articles, emails, and brainstorming ideas.

Limitations:

Free tier usage is capped; may require upgrading for heavy use.

Copy.ai

Copy.ai focuses on helping users craft engaging marketing copy, blog posts, and social media captions.

2. Image Generation Tools

DALL·EOpenAI’s DALL·E can generate stunning, AI-created artwork from text prompts. The free tier allows users to explore creative possibilities, from surreal art to photo-realistic images.

Craiyon (formerly DALL·E Mini)This free AI image generator is great for creating quick, fun illustrations. It’s entirely free but may not match the quality of professional tools.

3. Video Editing and Creation

Runway MLRunway ML offers free tools for video editing, including AI-based background removal, video enhancement, and even text-to-video capabilities.

Pictory.aiTurn scripts or blog posts into short, engaging videos with this free AI-powered tool. Pictory automates video creation, saving time for marketers and educators.

4. Productivity Tools

Notion AINotion's AI integration enhances the already powerful productivity app. It can help generate meeting notes, summarize documents, or draft content directly within your workspace.

Otter.aiOtter.ai is a fantastic tool for transcribing meetings, interviews, or lectures. It offers a free plan that covers up to 300 minutes of transcription monthly.

5. Coding and Data Analysis

GitHub Copilot (Free for Students)GitHub Copilot, powered by OpenAI, assists developers by suggesting code and speeding up development workflows. It’s free for students with GitHub’s education pack.

Google ColabGoogle’s free cloud-based platform for coding supports Python and is perfect for data science projects and machine learning experimentation.

6. Design and Presentation

Canva AICanva’s free tier includes AI-powered tools like Magic Resize and text-to-image generation, making it a top choice for creating professional presentations and graphics.

Beautiful.aiThis AI presentation tool helps users create visually appealing slides effortlessly, ideal for professionals preparing pitch decks or educational slides.

7. AI for Learning

Duolingo AIDuolingo now integrates AI to provide personalized feedback and adaptive lessons for language learners.

Khanmigo (from Khan Academy)This AI-powered tutor helps students with math problems and concepts in an interactive way. While still in limited rollout, it’s free for Khan Academy users.

Why Use Free AI Tools?

Free AI tools are perfect for testing the waters without financial commitments. They’re particularly valuable for:

Conclusion

AI tools are democratizing access to technology, allowing anyone to leverage advanced capabilities at no cost. Whether you’re a writer, designer, developer, or educator, there’s a free AI tool out there for you. Start experimenting today and unlock new possibilities!

4o

[shythemes]

hello ! im gonna pursue data science and i saw that we'll learn some coding in it so i just wanted to know ,, is coding very difficult?? do we need math for it ??

hi!

I’ve been working as a data scientist for 2 years, and I should probably note that I work at a traditional/non-tech company, I’m sure data scientists at tech companies have a very different experience

I majored in math in college and I got a masters in statistics, and I took only 2 courses in that time fully dedicated to coding, one was in C++ and the other was in SAS. several of my upper-level undergrad courses and all of my masters-level courses used some MATLAB, SAS, or R. I taught myself the basics of SQL and Python (mainly pandas and scikit-learn, two critical Python libraries for data science) before I applied for jobs because they came up a lot in job descriptions. I mentioned 6 coding languages there, but actually the only coding languages in my day-to-day work are SQL and Python, which I mostly taught myself

as you can see from this blog I’m not exactly a newbie to coding, and I remember coding CSS and HTML as early as age 10 or 11, so my experience may not be universal, but I don’t think coding is hard. in fact, coding will probably end up being your favorite part of your job as a data scientist. it beats attending meetings, preparing slides, writing emails, and (the worst of all personally) updating documentation

but math is one of the fundamental skills you will need as a data scientist, so just because you don’t necessarily need it for coding, you will probably need it for developing models, validating your models, and calculating key performance metrics

when I’ve interviewed candidates in the past, most will have a plethora of coding languages on their resume, but I’m only really looking for them to have a good grasp of SQL and Python if they have those on their resume. the thing that is harder to teach is the probability and machine learning theory

I hope I answered your question, sorry I probably wrote too much, but I enjoyed answering, thanks :)

[douchebagbrainwaves]

AND IT TURNS OUT TO HAVE SELFISH ADVANTAGES

We erred ridiculously far on the side while working on Wall Street. But the first is by far the biggest killer of startups that go public is very small. This can work well in technology, at least, that worry will now be out in the open instead of being a good writer than being a good speaker is not merely that it's longer. Shockley. But I don't write to persuade, if only out of habit or politeness. This habit is unconscious, but not smiling. Fundraising is brutal. We constantly have to make decisions about things we don't understand, and more often than not we're wrong.

These are smart people; if the technology was good, they'd have learned to ask that. When we interviewed programmers, the main thing we cared about was what kind of software that makes money and the kind that's interesting to write, and Microsoft's first product was one, in fact, but no rich people. If there is a lot more disagreeing going on, especially measured by the word. So why not go after corruption? The most important thing is not to change anyone's mind, but to reassure people already interested in using Lisp—people who know that Lisp is a computer language, and have a compiler translate it into machine language for you. Making things cheaper is actually a subset of a more general technique: making things easier to use. I heard there were about 20,000. As far as I know, no one of whom really owns it, it will disappear. It let them build great looking online stores literally in minutes. And to engage an audience you have to choose between two theories, prefer the one that doesn't feel mass-produced. Investors like it when you don't need it this month. Painters and writers notoriously do.

Whereas we felt pretty sure that we could sometimes duplicate a new feature within a day or two of a competitor announcing it in a press release. And compared to the facial expressions she was used to. You can work in plain sight and they don't realize the danger. Transposing into our original expression, we get: decreasing economic inequality means taking money from the rich. For a long time it was most of making things easier, but now that the things we could make sites for people who didn't want them to start treating us like actual consultants, and calling us every time they wanted something changed on their site. Something that curtly contradicts one's beliefs can be hard to sell. Building office buildings for technology companies won't get you a job, except perhaps as a classics professor, but it has more potential than they realize. VisiCalc made the Apple II. Our hypothesis was that if we wrote our software in a weird AI language, with a bizarre syntax full of parentheses. The cause must be external. What's special about startup ideas? Any advantage we could get in the way they framed the question.

Inexperienced founders make the same mistake when trying to convince investors of something very uncertain—that their startup will be huge—and convincing anyone of something like that except by implementing your way toward it. Instead of saying that your idea is to make money and maybe be cool, not to be too difficult for programmers used to C. Those companies were apparently willing to establish subsidiaries wherever the experts wanted to live. If you actually want to compress the gap between rich and poor, you have to sit with a teacup of types balanced on your knee and make polite conversation with a strict old aunt of a compiler. Right now the limiting factor on the number of investors just as we're increasing the number of desirable startups will probably grow faster than the percentage they sell to investors shrinks. True, but I don't think many people realize there is a lot of competition for a deal, the number that moves is the valuation and thus the amount invested. It has too many cooks. Whereas the bad firms will get the leftovers, as they do now, and yet you won't use it. If you can't find an actual quote to disagree with the author's tone. We expected the most common complaint you heard about Apple was that their fans admired them too uncritically. And it's not fun for a smart person to work in record stores.

Apple is trying to be a 2 week interruption turns into a 4 month interruption. Cobol is a high-level languages on the other. If they wanted Perl or Python. What was wrong with that? People will pay for? You can work in plain sight and they don't realize the danger. It seemed like selling out. This is the same argument you tend to hear for learning Latin. Whereas we felt pretty sure that we could hold our own in the slightly less competitive business of generating Web sites for art galleries. In fact, it's often better if they're not. But even if you only have one meeting a day with investors, somehow that one meeting will burn up your whole day. You must resist this.

Now Palo Alto is suburbia, but then it was a charming college town with perfect weather and San Francisco only an hour away. One big wave and you're sunk. For example, a politician announcing the cancellation of a government program will not merely say The program is canceled. This process is not just to baldly state the facts. If you're having trouble raising money from them is something that has to work very closely with a program written in a certain language, it might be a great thing. The five languages that Eric Raymond recommends to hackers fall at various points on the power continuum, he doesn't realize he's looking up. That's negligible as corporate revenues go, but the time getting there and back, and the most productive setup is a kind of thinking you do without trying to.

[tap-tap-tap-im-in]

A Web Development Master Post

I’ve spent the last two years working as a professional developer. I didn’t go to college for this, and just about everything I know I’ve either taught myself or learned from looking through other people’s source code as we research if we want to pull a project into our code base. I love it, and I have done some things I never would have expected from myself at the start. But before we get into any of those, I wanted to put together a list of resources I wish I had or worked with more fully when I was sitting in my job interview two years ago. Think of this as part resources on how to learn some of these skills, some recommendations on applications to incorporate into your workflow, and a few opinions on some of the other common applications that you’re welcome to heartily disagree with.

First things first lets get a few resources together, and for those of you who are already familiar with HTML, CSS, JavaScript, and PHP, none of these will be a surprise. It might be worth your while to jump ahead.

Online Resources

https://www.w3schools.com/

Starting out, W3 schools will probably be pretty omnipresent for help. They have tutorials for HTML, CSS, JavaScript, PHP, ASP, and many other web technologies. They pride themselves in being the largest web developer site, and unfortunately that has a downside. They don’t always update all of their articles to the most current specifications. This is a wonderful resource, as they do a good job of explaining a lot of these concepts in a beginner friendly way, but when you’re ready for nitty-gritty details, it’s almost always best to go with a more specialized developer resource.

https://css-tricks.com/

CSS is a powerful and flexible tool. Every day I see projects where developers have pushed it to new heights, but sometimes it’s a little arcane too. Well the wizards over at CSS-Tricks have collected a large number of articles and tutorials that explain everything from how z-index works to how to use newer layout-centric rules like Flexbox or Grid. If a CSS rule is misbehaving, 9 times out of 10, I can find a clear and concise reason on this site, and more importantly, I find many recommendations on how I can achieve the same effect differently.

https://flexboxfroggy.com/

Speaking of Flexbox, Flexbox Froggy is a one-note kind of resource, but it teaches all of the core concepts behind flexbox, and it can also teach CSS savvy managers why moving to a new layout methodology would benefit your work flow.

https://cssgridgarden.com/

Created by the same developer as Flexbox Froggy, and it does the same thing with Grid concepts.

https://developer.mozilla.org/en-US/

Mozilla’s developers have been at this game a long time, and their resources are next to none. In my opinion, this collection does not do much help a beginner understand, but the trade-off is that once you’re over that initial hurdle, the information you’re looking for is almost always only a few clicks away. This is the first, and often the last, place I go whenever I’m looking to solve a weird JavaScript bug that the rest of the internet is too clever to have encountered, (or to explain why only IE9 is seeing it).

http://php.net/manual/en/index.php

I know PHP isn’t a popular language right now, but it’s powerful, it’s flexible, and it’s still the primary language of the largest and most popular CMS on the planet, for better or worse, WordPress. Especially if you’re going to work freelance, you owe it to yourself to at least be familiar with PHP, and this will be your best friend. It’s no nonsense, and not beginner friendly, but it’s clear, and the comments on the articles are often as helpful as the articles themselves.

https://codex.wordpress.org/Developer_Documentation

Speaking of working in freelance (or even for a firm like I do), I have my own opinions about the way WordPress works, but you’re going to be doing projects in WordPress, and you aren’t going to be able to accomplish them without this. I have a small problem with the way functions and parameters are explained (it isn’t always easy to differentiate how one calls a function manually, or if it is called by filters, or how it is different from they three other functions named roughly the same thing), but I do know that the WordPress core developers work very hard, so there is always a method to the madness, even if you don’t have the key to see it.

https://stackoverflow.com/

When you get to the debugging stage, you’re going to become familiar with Stack Overflow pretty fast, as it almost always dominates the first few google results for a problem. Now, I’ve had developers try to scare me away from using Stack Overflow because it is open for beginners and experts, and sometimes it’s hard to tell quality of answers, but I strongly disagree with that. While it’s true you should always look cautiously at using someone else’s code right out of the box, there are a lot of members of the community that go out of their way to explain what the code is doing, and those are the answers you should be looking for. You don’t have to use their code, but if you can understand why you had the problem in the first place, you grow as a developer, and now have the tools to solve the problem. Stack Overflow is a big part of the Open Source community, so it’s always nice to give back at least as much as you take, so if you see a question you have the answer to, feel free to share.

https://github.com/

Eventually, you’re going to run into a project where you need a plugin developed by someone else, either because you don’t yet have the knowledge or you don’t have the time, but the client needs the functionality. 9 times out of 10 you’ll find what you need on GitHub, and honestly, you should be getting together your own GitHub with plugins and projects of your own. No matter how single purpose they may be, you’d be surprised how useful things can be in very specific situations. GitHub, like Stack Overflow, is a big part of the Open Source community, so it’s always nice to give back when you can. Make suggestions or report problems you have with any projects you pull, and in doing so you’ll make the community a better place.

Applications for Windows Based Developers

Now, for the next section let’s get into some tools. I love gaming, so I have a Windows computer at home. I don’t really know why we’re wasting money on a Windows license at work, but we are so I can mirror the full stack in both locations. Here’s what I use.

https://notepad-plus-plus.org/

When I first started, my boss insisted I use Dreamweaver for everything. I have nothing against Adobe, and their products are quality, but Dreamweaver was way more trouble than it was worth. Everything I cared about from Dreamweaver I can do in this free and Open Source program, with some extra functionality I find it extremely hard to mimic in Dreamweaver. Notepad++ is fast, stable, and hugely extendable. If you’re doing this as a full time job, I strongly recommend switching to a dark theme for the sake of your eyes. Blackboard is among my favorites (unless you’re trying to write Python). I also really love its macro functionality, I have a couple of re-used DOM structures programmed in there right now as well as my multi-line comment format.

https://winscp.net/eng/download.php

Arguably, the best part of Dreamweaver is the built in FTP client that lets you push changes directly to the server, but set Notepad++ as your primary editor in this, and suddenly you even have that feature, as this will sync temp folders back to the server. This is probably the best FTP client I’ve seen on Windows, with full support for SFTP and SSH (built on PuTTY) with all sorts of encryption and authentication options. It’s also hugely configurable and fairly dependable. If you’re working on a remote server using a Windows machine, this is probably how you should be accessing the file system.

https://www.putty.org/

I hope that as a web developer you don’t have to learn to be a server administrator, but as a web developer I am telling you you’ll probably have to learn at least a little bit about Unix/Linux server administration, as they are by far the most popular web server stacks out there, and you’ll be controlling them with an SSH client at some points, even if a web interface is available. This is a great one, with all sorts of authentication options, so if (like us) you know you need root access to a server remotely, but you don’t like the idea of protecting that with just a simple password, you can set up Public/Private key pairs with encryption passphrases.

http://www.wampserver.com/en/

Let me be clear on this: WAMP, which stands for Windows Apache, PHP, MySQL, is great to have. It’s good for training, it’s good for prototyping tools without having to wait for a virtualized server to start up, but the differences in environments between running Apache, PHP, and MySQL on Windows versus Linux will bite you eventually. Don’t expect to be able to push anything you worked on in WAMP directly to your Linux based server without having to fix a few problems here and there. That said, I have a number of things I run in my WAMP server all the time (linting, IP geolocation, domain DIGs, and a few others). It’s a great tool, but it isn’t a replacement for a staging server.

https://www.virtualbox.org/

You should be using a virtual machine for your staging server. That way you can simulate things like network communications and how your code will actually be run on Linux. Virtual Box is free and powerful. You’ll need to get ISOs for whatever operating system you intend to run, and you should be aware that at least some versions of Windows have it written into their License agreement that you can’t run them virtualized.

https://www.gimp.org/

Gimp is powerful. I don’t really know how to use it well. It’s always been one of those things where I know I need to sit down and teach myself, but since I’ve fallen far into the trenches of server backend work, I haven’t ever had the motivation. Mostly, I use this to resize images when I notice that a website is loading a 14MB PNG on the homepage for some unknown reason. Please designers, think of the mobile phones, keep total page loads (Including all resources, pictures, scripts, and DOM structure) as close to or below 1MB as you can, especially if your site is supported by ad revenue, as there’s no telling how much the ads will need to load on top of that.

https://tools.stefankueng.com/grepWin.html

GrepWin is an implementation of Grep functionality on Windows. For those of you who aren’t aware, Grep is a terminal tool on Linux/Unix that uses a very efficient algorithm to search through large amounts of text for whatever you define, be it flat text, or something represented by a regular expression. It’s super useful for renaming an included document or global variable, and can really save your bacon if your error reporting is being vague. I like this particular implementation because it has context menu integration, so it’s as easy to use as right clicking in the directory you want to search in and telling it to search. It also supports text replacement with backups, so this simple tool is extremely useful more often than I’d like to admit.

https://gitforwindows.org/

The last tool is an implementation of Git for windows that also includes a Bash terminal. This is important because a lot of developers work in Linux, and so installation directions might only be available as Bash code, this makes it easy to move past that step without being bogged down translating that into Windows CMD code. This is a full implementation of Git, so it comes with all of the version control features and easy project building that Git provides. If you end up working with Electron or Node.JS in general, you’ll end up leaning on this pretty hard.

And that’s it. I’m hoping that later this week I can get into more interesting stuff, but I wanted to have a foundational post of the resources I might reference and the tools I’m using for people to fall back on. It’s the kind of thing I wish I had to reference when I was starting out, especially since all of the tools I’m using now are free and Open Source, and making that change has sped up my workflow significantly, as the only application I’ve listed here with any noticeable boot time is Gimp, something I hated about Dreamweaver every time I had to shut the computer down for whatever reason.

I plan on coming back to this post periodically and updating it as my opinions change, or I become aware of other resources that should be on here. Eventually I’ll also be lining out a software for Linux section, but I’m still shopping around for an affordable and stable Linux development machine.

Next time I think we’re going to dive right into some anecdotes about code commenting and design patterns, and why it pays to think about those from the beginning. Nothing glamorous, but I’d argue hugely important, and you get to laugh with me about some dumb things I’ve done.

[superlinguo]

Linguistics jobs -  Interview with a Linguistic Project Manager at a Language Tech Company

This month’s interview highlights the increasingly central role of human language in tech, and the important role that linguists can play in the current tech landscape. It’s also been a delight for me to interview Sasha Wilmoth, who I last caught up with when she was taking one of her final undergraduate classes at Melbourne Uni. This interview is also something  slightly different, because Sasha is one of a growing number of ‘hybrid’ researchers - working across both the university and private sector. I’ll leave it to her to explain more!

What did you study at university?

I did a Bachelor of Arts majoring in Linguistics, and studied Mandarin and Swedish. I also squeezed in as many literature subjects as I could. Within linguistics, I became interested in the morphology and syntax of Australian Aboriginal languages – my honours thesis was on Murrinhpatha, which is a polysynthetic language spoken in the Northern Territory.

What is your job?

That’s a good question! I think most of my family and friends still don’t really understand what I do for a living.

My time has been split between two jobs this year: I’m a Linguistic Project Manager at a company called Appen, and a Research Assistant within the ARC Centre of Excellence for the Dynamics of Language (CoEDL). I’ll just focus on my job at Appen, because it’s a bit further away from academia.

Basically, Appen provides all sorts of data for AI and machine learning, including for speech and language technology. In the Linguistic Services department, we make the specialised annotated data that’s used to train these systems, in whatever language our clients need (we’re up to something like 180 languages and dialects by now). This includes pronunciation lexicons, prosodic annotation, part-of-speech tagging, tokenisation, proofing tools, guidelines, and so on. There are also lots of linguists working in the Data Collection, Transcription, and Translation departments. My job encompasses the technical and linguistic bits, as well as the more typical project management things like planning, budgeting, and delegating tasks.

For example, for a pronunciation lexicon project, which is what I’ve largely been working on, I start by researching the grammar, phonology, orthography and dialects of the language. Based on this research, I figure out what our approach to phonemic transcription will be and come up with some technical tools to predict the pronunciation of the words based on the orthography. I manage a team of linguists around the world who are native speakers of that language, and they check that the pronunciations are correct. Then I do a bunch of quality checks according to what we know about the language and prepare the data for our clients. Recently, I’ve been working on Serbo-Croatian, Persian, and Mandarin projects – as you can imagine, each language presents completely different technical and linguistic challenges.

Appen is an industry partner with CoEDL, so another cool thing I get to do is collaborate with academic linguists and see if we can apply our own processes and problem-solving skills to their research projects.

How does your linguistics training help you in your job?

I’m happy to say that I apply my linguistics education at work every day: I solve linguistic puzzles and read grammars and write phonological rules and use ToBI (a way of transcribing intonation) and the International Phonetic Alphabet. I never managed to become a very fluent Mandarin speaker, but I use that knowledge for our Mandarin and Cantonese projects. I even do a bit of Swedish proof-reading here and there. I didn’t imagine this was possible when I chose this rather impractical combination of subjects.

Do you have any advice you wish someone had given to you about linguistics/careers/university?

I know some previous interviewees have said they wish they had been more aware of the (lack of) job prospects for linguists. I’m stubborn though, and I was determined to study what I was most interested in without taking into account career prospects. I’m really glad I made that choice (and had the privilege to do so), because there are actually more opportunities outside of academia than I thought. The future for linguists in industry is bright, as we’ll be using speech more and more to interface with our phones, cars, toasters, personal robot servants, etc.

Most linguistics students don’t end up working as linguists, so it’s wise to keep your options open. But if you’re motivated and can’t think of anything else you’d rather be doing, there are jobs out there – I wish I had known about them when I was at uni. It’s really important to seek out and take advantage of opportunities for students like summer research programs, internships, or volunteering with organisations like RNLD here in Melbourne. Get to know your lecturers and tutors and ask them what you can get involved in.

Any other thoughts or comments?

Just a note on working in tech: I don’t have a computer science background, and I’ve never been a big computer nerd. I’ve learned everything on the job, and it turns out I quite like computers after all. If you have the type of pattern-finding brain that linguists tend to have, chances are you might enjoy the technical side of things when it’s applied to something interesting. Developing some technical skills as a student can help you in many career paths, but it’s not an absolute necessity. If you’re looking for a place to start: regular expressions, bash, and Python are all very useful and relatively easy to learn.

Previously:

Interview with a Data Scientist

Interview with a Librarian

Interview with a Text Analyst

Interview with a User Experience (UX) Researcher

Interview with a Study Abroad Facilitator

Interview with The Career Linguist

Interview with a local radio Digital Managing Editor

Check out the Linguist Jobs tag for more interviews

[fastforwardlabs]

Thomas Wiecki on Probabilistic Programming with PyMC3

A rolling regression with PyMC3: instead of the regression coefficients being constant over time (the points are daily stock prices of 2 stocks), this model assumes they follow a random-walk and can thus slowly adapt them over time to fit the data best. 

Probabilistic programming is coming of age. While normal programming languages denote procedures, probabilistic programming languages denote models and perform inference on these models. Users write code to specify a model for their data, and the languages run sampling algorithms across probability distributions to output answers with confidence rates and levels of uncertainty across a full distribution. These languages, in turn, open up a whole range of analytical possibilities that have historically been too hard to implement in commercial products.

One sector where probabilistic programming will likely have significant impact is financial services. Be it when predicting future market behavior or loan defaults, when analyzing individual credit patterns or anomalies that might indicate fraud, financial services organizations live and breathe risk. In that world, a tool that makes it easy and fast to predict future scenarios while quantifying uncertainty could have tremendous impact. That’s why Thomas Wiecki, Director of Data Science for the crowdsourced investment management firm Quantopian, is so excited about probabilistic programming and the new release of PyMC3 3.0.

We interviewed Dr. Wiecki to get his thoughts on why probabilistic programming is taking off now and why he thinks it’s important. Check out his blog, and keep reading for highlights!

A key benefit of probabilistic programming is that it makes it easier to construct and fit Bayesian inference models. You have a history working with Bayesian methods in your doctoral work on cognition and psychiatry. How did you use them?

One of the main problems in psychiatry today is that disorders like depression or schizophrenia are diagnosed based purely on subjective reporting of symptoms, not biological traits you can measure. By way of comparison, imagine if a cardiologist were to prescribe heart medication based on answers you gave in a questionnaire! Even the categories used to diagnose depression aren’t that valid, as two patients may have completely different symptoms, caused by different underlying biological mechanisms, but both fall under the broad category “depressed.” My thesis tried to change that by identifying differences in cognitive function -- rather than reported symptoms -- to diagnose psychiatric diseases. Towards that goal, we used computational models of the brain, estimated in a Bayesian framework, to try to measure cognitive function. Once we had accurate measures of cognitive function, we used machine learning to train classifiers to predict whether individuals were suffering from certain psychiatric or neurological disorders. The ultimate goal was to replace disease categories based on subjective descriptions of symptoms with objectively measurable cognitive function. This new field of research is generally known as computational psychiatry, and is starting to take root in industries like pharmaceuticals to test the efficacy of new drugs.

What exactly was Bayesian about your approach?

We mainly used it to get accurate fits of our models to behavior. Bayesian methods are especially powerful when there is hierarchical structure in data. In computational psychiatry, individual subjects either belong to a healthy group or a group with psychiatric disease. In terms of cognitive function, individuals are likely to share similarities with other members of their group. Including these groupings into a hierarchical model gave more powerful and informed estimates about individual subjects so we could make better and more confident predictions with less data.

Bayesian inference provides robust means to test hypotheses by estimating how different two different groups are from one another. 

How did you go from computational psychiatry to data science at Quantopian?

I started working part-time at Quantopian during my PhD and just loved the process of building an actual product and solving really difficult applied problems. After I finished my PhD, it was an easy decision to come on full-time and lead the data science efforts there. Quantopian is a community of over 100.000 scientists, developers, students, and finance professionals interested in algorithmic trading. We provide all the tools and data necessary to build state-of-the-art trading algorithms. As a company, we try to identify the most promising algorithms and work with the authors to license them for our upcoming fund, which will launch later this year.  The authors retain the IP of their strategy and get a share of the net profits.

What’s one challenging data science problem you face at Quantopian?

Identifying the best strategies is a really interesting data science problem because people often overfit their strategies to historical data. A lot of strategies thus often look great historically but falter when actually used to trade with real money. As such, we let strategies bake in the oven a bit and accumulate out-of-sample data that the author of the strategy did not have access to, simply because it hadn’t happened yet when the strategy was conceived. We want to wait long enough to gain confidence, but not so long that strategies lose their edge. Probabilistic programming allows us to track uncertainty over time, informing us when we’ve waited long enough to have confidence that the strategy is actually viable and what level of risk we take on when investing in it.

It’s tricky to understand probabilistic programming when you first encounter it. How would you define it?

Probabilistic programming allows you to flexibly construct and fit Bayesian models in computer code. These models are generative: they relate unobservable causes to observable data, to simulate how we believe data is created in the real world. This is actually a very intuitive way to express how you think about a dataset and formulate specific questions. We start by specifying a model, something like “this data fits into a normal distribution”. Then, we run flexible estimation algorithms, like Markov Chain Monte Carlo (MCMC), to sample from the “posterior”, the distribution updated in light of our real-world data, which quantifies our belief into the most likely causes underlying the data. The key with probabilistic programming is that model construction and inference are almost completely independent. It used to be that those two were inherently tied together so you had to do a lot of math in order to fit a given model. Probabilistic programming can estimate almost any model you dream up which provides the data scientist with a lot of flexibility to iterate quickly on new models that might describe the data even better. Finally, because we operate in a Bayesian framework, the models rest on a very well thought out statistical foundation that handles uncertainty in a principled way.

Much of the math behind Bayesian inference and statistical sampling techniques like MCMC is not new, but probabilistic tooling is. Why is this taking off now?

There are mainly three reasons why probabilistic programming is more viable today than it was in the past. First is simply the increase in compute power, as these MCMC samplers are quite costly to run. Secondly, there have been theoretical advances in the sampling algorithms themselves, especially a new class called Hamiltonian Monte Carlo samplers. These are much more powerful and efficient in how they sample data, allowing us to fit highly complex models. Instead of sampling at random, Hamiltonian samplers use the gradient of the model to focus sampling on high probability areas. By contrast, older packages like BUGS could not compute gradients. Finally, the third required piece was software using automatic differentiation -- an automatic procedure to compute gradients on arbitrary models.

What are the skills required to use probabilistic programming? Can any data scientist get started today or are there prerequisites?

Probabilistic programming is like statistics for hackers. It used to be that even basic statistical modeling required a lot of fancy math. We also used to have to sacrifice the ability to really map the complexity in data to make models that were tractable, but just too simple. For example, with probabilistic programming we don’t have to do something like assume our data is normally distributed just to make our model tractable. This assumption is everywhere because it’s mathematically convenient, but no real-world data looks like this! Probabilistic programming enables us to capture these complex distributions. The required skills are the ability to code in a language like Python and a basic knowledge of probability to be able to state your model. There are also a lot of great resources out there to get started, like Bayesian Analysis with Python, Bayesian Methods for Hackers, and of course the soon-to-be-released Fast Forward Labs report!

Congratulations on the new release of PyMC3! What differentiates PyMC3 from other probabilistic programming languages? What kinds of problems does it solve best? What are its limitations?

Thanks, we are really excited to finally release it, as PyMC3 has been under continuous development for the last 5 years! Stan and PyMC3 are among the current state-of-the-art probabilistic programming frameworks. The main difference is that Stan requires you to write models in a custom language, while PyMC3 models are pure Python code. This makes model specification, interaction, and deployment easier and more direct. In addition to advanced Hamiltonian Monte Carlo samplers, PyMC3 also features streaming variational inference, which allows for very fast model estimation on large data sets as we fit a distribution to the posterior, rather than trying to sample from it. In version 3.1, we plan to support more variational inference algorithms and GPUs, which will make things go even faster!

For which applications is probabilistic programming the right tool? For which is it the wrong tool?

If you only care about pure prediction accuracy, probabilistic programming is probably the wrong tool. However, if you want to gain insight into your data, probabilistic programming allows you to build causal models with high interpretability. This is especially relevant in the sciences and in regulated sectors like healthcare, where predictions have to be justified and can’t just come from a black-box. Another benefit is that because we are in a Bayesian framework, we get uncertainty in our parameters and in our predictions, which is important for areas where we make high-stakes decisions under very noisy conditions, like in finance. Also, if you have prior information about a domain you can very directly build this into the model. For example, let’s say you wanted to estimate the risk of diabetes from a dataset. There are many things we already know even without looking at the data, like that high blood sugar increases that risk dramatically -- we can build that into the model by using an informed prior, something that’s not possible with most machine learning algorithms.

Finally, hierarchical models are very powerful, but often underappreciated. A lot of data sets have an inherent hierarchical structure. For example, take individual preferences of users on a fashion website. Each individual has unique tastes, but often shares tastes with similar users. For example, people are more likely to have similar taste if they have the same sex, or are in the same age group, or live in the same city, state, or country. Such a model can leverage what it has learned from other group members and apply it back to an individual, leading to much more accurate predictions, even in the case where we might only have few data points per individual (which can lead to cold start problems in collaborative filtering). These hierarchies exist everywhere but are all too rarely taken into account properly. Probabilistic programming is the perfect framework to construct and fit hierarchical models.

Interpretability is certainly an issue with deep neural nets, which also require far more data than Bayesian models to train. Do you think Bayesian methods will be important for the future of deep learning?

Yes, and it’s a very exciting area! As we’re able to specify and estimate deep nets or other machine learning methods in probabilistic programming, it could really become a lingua franca that removes the barrier between statistics and machine learning, giving a common tool to do both. One thing that’s great about PyMC3 is that the underlying library is Theano, which was originally developed for deep learning. Theano helps bridge these two areas, combining the power nets have to extract latent representations out of high-dimensional data with variational inference algorithms to estimate models in a Bayesian framework. Bayesian deep learning is hot right now, so much so that NIPS offered a day-long workshop. I’ve also written about the benefits in this post and this post, explaining how Bayesian methods provide more rigor around the uncertainty and estimations of deep net predictions and provides better simulations. Finally, Bayesian Deep Learning will also allow to build exciting new architectures, like Hierarchical Bayesian Deep Networks that are useful for transfer learning. A bit like the work you did to get stronger results from Pictograph using the Wordnet hierarchy.

Bayesian deep nets provide greater insight into the uncertainty around predicted values at a given point. Read more here. 

What books, papers, and people have had the greatest influence on you and your career?

I love Dan Simmons’ Hyperion Cantos series, which got me hooked on science fiction. Michael Frank (my PhD advisor) and EJ Wagenmakers first introduced me to Bayesian statistics. The Stan guys, who developed the NUTS sampler and black-box variational inference, have had a huge influence on PyMC3. They continue to push the boundaries of applied Bayesian statistics. I also really like the work coming out of the labs of David Blei and Max Welling. We hope that PyMC3 will also be an influential tool on the productivity and capabilities on data scientists across the world.

How do you think data and AI will change the financial services industry over the next few years? What should all hedge fund managers know?

I think it’s already had a big impact on finance! And as the mountains of data continue to grow, so will the advantage computers have over humans in their ability to combine and extract information out of that data. Data scientists, with their ability to pull that data together and build the predictive models will be the center of attention. That is really at the core of what we’re doing at Quantopian. We believe that by giving people everywhere on earth a platform that’s state-of-the-art for free we can find that talent before anyone else can.