dbaOnTap @dbaontap - Tumblr Blog

#OnTapToday Agentic Data Foundry, Part 3: The Trust Layer — Making Agentic Output Production-Grade

View On WordPress

Agentic Data Foundry, Part 2: The Deployment Problem

#OnTapToday — This is Part 2 of a series. Part 1 introduced the Agentic Data Foundry: the idea that data engineers should stop writing transformation code and start describing intent — letting AI agents handle the SQL. If you haven’t read that, it’s worth a few minutes. This post asks the question that every engineer eventually asks after seeing the demo: “Okay, but how do you actually ship…

View On WordPress

Agentic Data Foundry, Part 2: The Deployment Problem #OnTapToday — This is Part 2 of a series. Part 1 introduced the Agentic Data Foundry: the idea that data engineers should stop writing transformation code and start describing intent — letting AI agents handle the SQL. If you haven’t read that, it’s worth a few minutes. This post asks the question that every engineer eventually asks after…

View On WordPress

Fun Refactoring Python Code

I recently came across a coding challenge. Well, let’s say we needed to handle several combinations of filtering data in a #Pandas Dataframe leveraging the #Streamlit selectbox widget. Not a difficult lift per se. The fun was where it started and where it ended up. Let’s get started. Scenario I have 4 columns of data that I need to be filtered. Let create a sample dataset with the following…

View On WordPress

Snowflake JAVA UDF with JAR - Streamlit bonus

Snowflake JAVA UDF with JAR – Streamlit bonus

Recently, I was asked the following question: “How can I manage my Snowflake JAVA UDFs without needing to re-create the UDF every time I make a change to the logic? And by the way can you show me how to invoke that from Streamlit? The first thing that popped in my head was, “Streamlit is a Python-based library, so how do we ‘call’ a JAVA program?” This was short lived, but yeah. The steps below…

View On WordPress

Data Warehouse Migration with Snowflake VSC and Flyway

Wow! It has been a year since my last blog post. This post was originally going to cover getting #Flyway set up with Snowflake. Then a couple of days ago, my friend Kent Graziano tweeted a link on getting Visual Studio Code (VSC) connected to Snowflake. Considering I had just come off a lengthy project where VSC was the IDE of choice, I figured, what the heck let’s get this set up. I’m sure to…

View On WordPress

Let DBeaver Build Your ERD from a Snowflake Data Warehouse

I was recently responding to a question about DBeaver and the ability to automatically generate an ERD. The following post will walk you through the process.

;TLDR Looking for an easy way to create one of these?

Let’s create 3 simple tables. A parent table with a primary key constraint and two additional tables with foreign key constraints that point to the same columns as the parent…

View On WordPress

#DBeaver #ERD

Python Abort trap: 6 fix after #Catalina update

#OnTapToday Python Abort trap: 6 fix after #Catalina update

A couple of days ago, I update my MacOS to Catalina and immediately after that, all the Snowflake scripts I wrote in #Python started returning an Abort trap: 6 error. My initial thought was something must be a compatibility issue with either the python connector or new default terminal shell zsh. Fortunately I was wrong, but here is what I found and how i was able to do get my Python scripts…

View On WordPress

#Troubleshooting

Snowflake Migration using StreamSets

#OnTapToday, using @StreamSets to migrate data from your source database to Snowflake. In this post, I will walk through the process of configuring StreamSets to migrate data from my on prem DB to Snowflake.

WHY

I needed to get some data that I was working on from my Oracle database to the Snowflake data warehouse. There were a few options at my disposal. I could have:

done a simple export of the…

View On WordPress

#Migration #StreamSets

The Winds of Change

I was asked a while ago by a twitter follower this question. “One year ago to now, how has your use of technology changed in your job role?” Interestingly enough, I celebrated 1 year @snowflakeDB on April 16, 2019. I think now is a good time to answer that question.

In a nutshell, a year ago, I was 100% focused on building out private cloud infrastructure on a single platform. If you have…

View On WordPress

Connect to Snowflake Data Warehouse with GO

In this installment, I am going to walk through the process of connecting GO to the Snowflake Data Warehouse Service (DWaaS). This tutorial requires that you have a Snowflake account. You can sign up here for a 30 day/$400.00 trial.

Download and Install the ODBC Driver

Once you have your account set up in Snowflake, follow the directions to download the ODBC Driver. It will the same place you…

View On WordPress

#golang

ODTUG Kscope18 in the bag

One month ago, ODTUG Kscope18 was getting started, and one month ago was the first time in two years that I was not the Conference Chair, and today, I can say that my friend Opal did a damn good job at taking the ODTUG Kscope conference to the next level. Thank you Opal for your leadership and making this Kscope, like all others before it, “The best one ever!!”

As with each ODTUG Kscope, we…

View On WordPress

#Conference #Kscope18

Connect to Snowflake with Node.js Driver

This will be the first entry into my Snowflake toolbox. Today I am going to set my MacBook Pro up to connect to a Snowflake data warehouse via the Snowflake Node.js driver. This could come in handy when needing to move data between systems where direct connections are not available and you need an intermediary. Let’s get started.

This tutorial does require you to have access to a Snowflake data…

View On WordPress

#Nodejs #Snowflake

Become the Change You Seek!

By the time this posts hits, I’ll be in the air heading to one of my favorite cities, San Francisco. — My Sifu, Coach Sule, has that slogan as part of his email signature, and those are words are quite fitting. If you have been following me on Twitter, then you know that a lot is going on in the life and times of @dbaOnTap. The blog title definitely says it all and change is a comin’!!

Have…

View On WordPress

#Data Warehouse #Docker Captain #Oracle Ace

Become the Change You Seek

My Sifu, Coach Sule, has that slogan as part of his email signature, and those are words are quite fitting. If you have been following me on Twitter, then you know that a lot is going on in the life and times of @dbaOnTap. The blog title definitely says it all and change is a comin’!! Have you ever thought to yourself, life is great! I wouldn’t change a thing! How quickly those things can…

View On WordPress

Case Insensitivity with JSON and Oracle 12c?

About a week ago, I was presenting at ECO on JSON parsing in the Oracle 12c Database and an attendee asked a great question. To summarize, Maria Colgan gave a keynote in which she noted a feature in 12c that allows you to define a column as Case Insensitive. The attendee in my session asked if this also applied to JSON in the Oracle database.

There are enough posts on how to configure your…

View On WordPress

#12c #JSON #Oracle

ODC Appreciation Day : SQL Developer

Not sure if you’ve noticed, but there have been a few changes in Oracle’s advocacy community. The Oracle Technology Network has been rebranded as The Oracle Developer Community (ODC), and the Oracle ACE and Java Champions have a new baby sibling, called the Developer Champion. In the 2nd, hopefully annual Appreciation Day (started by Tim Hall), the community is releasing a series blog posts…

View On WordPress

Trending Blogs

Recently Viewed Blogs

dbaOnTap