Beetle Blog

[beetlemail-blog]

New feature: Daily subscriptions emails

You want to get the latest emails from your subscriptions daily! Weekly isn’t enough. As a marketing specialist you want to know what is happening in your industry and with your competitors each day, and you want it in your inbox ready for you each morning.

We’ve made that happen!

You can now get a daily email summarising the latest creatives from your Beetle subscriptions.

Go to your account settings:

You’ll see 3 options, Profile, Notifications and Contact Us. Select Notifications:

Here you can then set up to receive summary emails once a day.

Let us know if you need help setting this up on your account.

[beetlemail-blog]

New feature: Sender Name ‘Fuzzy Matching’

We had some great feedback about Sender Name Fuzzy Matching. This is where you can collate all of the emails for a certain term, even if the Sender name can vary slightly.Previously you had to set up subscriptions for different senders / names to capture the full range of emails sent by some larger brands, for example Adidas.

We decided to make things LOADS easier and allow for you subscribe using Fuzzy Matching on a term, collating all of the emails that have been sent by a sender name containing that term. You can see here with the example of Adidas, who send emails under a variety of names, you now can get all emails containing the word “Adidas” even if the sender name would normally appear as “Adidas”, “Adidas Online Shop”, “Adidas x Zalando” etc.

You can search using Fuzzy Matching in the Advanced Search function.

The Fuzzy Matching allows for you to be able to see all the email creatives you were looking for related to the term you are searching for:

And all of the analytics related to the term “Adidas”:

We are really pleased with this, we hope you are too!!

[beetlemail-blog]

Case Study#1: Fashion pt. 2

Timing.

One of my favourite features of Beetle Analytics is the time tracker of when emails are sent out in marketing campaigns. 

In our fashion research there are two very distinct target audiences, 1, ‘the early birds’ and 2, ‘the late risers’. 

‘The early birds’, this is where an email is sent first thing in the morning, generally between 6am and 10am. ASOS are a prime candidate of predominantly being an ‘early bird’ marketer:

You can see from the graph above 73% of their emails are sent at either 7am, 8am or 9am. This is because they want to reach their audience first thing, when they wake up and quickly check their emails on their phone - giving influence to their style of the day, or when you first get to the office and trawl through the mass of emails you were sent over night, probably mostly from Groupon, I know mine are. Here they can appeal to you as a distraction from the admin that everyone starts the day with, or if you didn’t manage to nail your outfit quite as well as you hoped, they have a range of alternatives available at the click of a button.

The timing here indicate their target audience is that of young professional who find that small window of time to check in on the latest ASOS news, either on their commute or first getting to work.

‘The late risers’ are represented by brands like Missguided who send 46%, their largest group of their emails between 9am and 12pm:

This reflects the younger audience they are targeting, they will catch the professionals on their lunch break, but the timings coincide with the later rising young adult they target. 

At Beetle we have email marketing analytics data back to 2013, so I ran the analytic specific to the “Fashion” Industry and found that 51% of emails within fashion are targeted as  ‘the late risers’. Not what I expected from the state of my inbox in the morning. But it makes complete sense with it being a time of the day where the majority of people are likely to be able to check their emails - lunch breaks at work, school, a bit of a break for homemakers or the lucky people who are gifted with a lazy morning:

[beetlemail-blog]

Case Study #1: Fashion

Sales Vs. Story: The Balance of Content

Due to the nature of the products being promoted the emails for the Fashion industry are primarily image based. However there is a stark contrast in the imagery used. The brands that target a younger audience often include more colourful imagery, with bright colours used for backgrounds and models wearing the product with some limited text. 

The more ‘premium’ brands tend to use a monochrome based colour palette, with simple block lines and text.

Emails targeted at women use very simple text and lots of whole body shots, whereas the ones targeting men focus on the actual items rather than the whole model. The is also usually an additional article or hook which tends to be an exclusive music track or interview. The phrasing between the gender's emails is also different. Female targeted emails assume that women know what they are looking for and are more savvy The male focussed campaigns are more questionable, reasoning with the audience in a “Hey, we understand it is difficult, let us help you out” kind of way.

A definite point of note is the tone of voice and how it varies so much between brands, some brands like Topman, Topshop and Vans tend to speak to their audience, almost filling a magazine type role. This is reflective of the lifestyle brands they aim to be and shows a great understanding for their customer base. 

Other brands like Treds or The Gap are very much “Sale! Sale! Sale!” - an almost lazy approach to speaking to their audience. It is only an opinion, but I feel it lessens the quality of the brand and doesn’t really compliment with their in-store experience, more specifically with The Gap, classically known for when you get your basics from - jeans, plain t shirts, but good quality simple stuff. This is something I feel they could really play upon.

Starting at the top of the experience with the Subject Lines, these brands the push the sales do exactly the same in their subject lines - Dorothy Perkins are a prime example of blasting the Sale across their opening gambit. If you check out the Dorothy Perkins analytics page for their Top Subject Line Keywords, you’ll see the following:

Those who are more in tune with their customer base  - or at least hold a dialogue with them, then use conversation starters as their Subject Lines ie “Iceland’s Greatest Export?” and with the nature of this client, the audience will know it may be something of interest, or at least it will incite some curiosity to open the email.

That being said the most common chart topping keyword in subject line analytics for fashion retailers is most definitely “Sale”. But Here you can see Selfridges & Co do their best to connect with their audience on a range of topics.

On that note we will leave it there for this chapter, but here is my favourite email’s to receive from the premium surf brand Finisterre who really do try to connect with their audience in their ‘Sunday Broadcast’ emails (there is no mention of a sale anywhere there, just glorious content!)

[beetlemail-blog]

Case Study Series

We often get asked by Beetle users to help them get started collecting insight on particular brands or industries, so here at Beetle we've decided to create a few case studies on some of the more popular requests. 

The first of these will be ‘Fashion’.

These case studies are pretty large, so we’ll be posting small sections to give you insight into areas that might relate best to what you are looking for.

Here we go… (here is a generic corporate image)

[beetlemail-blog]

New feature: Spam Scores!

We have another new feature to announce to add to our recently released industry classification: Spam scores provided by a fresh install of SpamAssassin. If you’re not aware, SpamAssassin is a widely used, open source spam filter which gets automatically updated with new rules based on information it gathers from mailservers around the globe. 

We’re in the process of rolling spam scores out to all our emails. If a spam score is available for an email, it will be shown on the left hand side when in the email view. See the screenshot below for an example. 

The default configuration of SpamAssassin considers any score above a 5.0 to be spam. This will be reflected in the box under the score, for the information at a glance. 

You can also see the average spam score for your search in the analytics panel! Check out the next screenshot to see what that might look like.

While the spam scores are still being worked out, you may see some emails without a spam score. These will be added soon.

Stay tuned and stay spam free 😉

[beetlemail-blog]

Feature Highlight: Suggest a site

We at Beetle are keen to constantly hear about improvements and suggestions our users have. One obvious area open to suggestions is the actual content we have in our database.

We constantly strive to have the best and most up to date information for you guys to use, but sometimes a little pointing in the right direction helps us out a lot!

If we don’t have a brand or campaign you are looking for from a specific site you can “Suggest a site” so we can get signed up to their email cycle and begin collecting analytics and email creative for you.

It’s really simple and really quick!

All you need to do is click in the side menu on the “Suggest a site” option that looks like this -

If you click on this option you will have a window pop up asking you to enter the website url you would like us to sign up to for you:

If you enter the url as shown above and click submit, Beetle will automatically try and sign up to that site’s email campaigns as soon as possible!

Teamwork! :)

[beetlemail-blog]

New feature: Advanced search with industry classification!

Today we unveiled something we’ve been working very hard on, and are very excited about; our advanced search.

We have offered search filtering for a while, but it was only exposed by using some admittedly ambiguous search terms. We’ve revised our search functionality and have made it a whole lot easier to find exactly what you want.

Under your trusty old search box, you’ll now see an “advanced search”  option. Clicking this will present you with a plethora of filters you can use to whittle down your search.

Possibly the most exciting piece of this new feature however, is our new industry classification! Now you can filter your emails by the industry you’re interested in. An in depth how to guide will be coming soon, but check out the screenshots below to get an idea of what we’ve got waiting for you. 

Be aware that the industry classification feature is very much in beta and as such a work in progress. This means that you may see the occasional incorrectly classified email in your searches. If you like, you can let us know about these emails via the “Report a problem” button when viewing an email.

If you’re wondering how on earth we could incorrectly classify an email, it’s because we’re developing a system that will look at the content of an email and decide which industry it belongs to, based on its keywords, who it’s from and the website it was received from. We’ve flicked the “automatically get better over time” switch on, and it should get better at this classification the more emails it sees.

As well as the in depth guide to our advanced search, for all you techies we’ll be writing a high level view of how we’ve achieved this. Here at Beetle, we’re tech-heads at heart, so we can’t wait to tell you more!

[beetlemail-blog]

How To Guide: Creating Collections

Collections are one of my favourite features of Beetle (definitely biased!). they allow for you to, you guessed right, create collections of your favourite search results. 

As the English Premier League has recently begun for another year, I was curious to see what build-up there had been in the email world over the past few months and save my research into my own “Football” Collection from the position of a sports brand. 

Step 1: Start your search

I started with the basics and searched for the term “football”

This presented me with the creatives for every single email we have related to the term “football”.

Step 2: Refine your search results

The next step was the refine the search results. As i am interested in the build-up to the English Football season 2016/17, I reduced my email results to only show since May 2016, making them more time relevant.

Here are the results I was presented. As I am interested in how clubs and brands were marketing, I chose not to focus on some of the text heavy malling list. 

Step 3. Adding to your Collection

Pro Direct Soccer is a brand I felt relevant to the type of research I was conducting, so I clicked on their email creative.

Eureka! This is the type of content I want for my research and I definitely want to include this in my Collection.I clicked on the Collections icon in the corner of the screen.

When you click on this icon a modal will pop-up allowing for you to add the email to an existing Collection or start a new one.

If I save this new Collection, I can perform multiple searches and add to it as and when I come across emails that I want to keep. 

Step 4. Revisiting your Collection

In the sidebar menu I can select the same Collections icon and refer back to all of my collections at any time. 

To see the contents of my Collection, I simply need to click on the grey box. Inside I can see all of my saved emails for this piece of research to date.

Your Collections will consist of only the emails you have saved and these will not expire, so you can refer back to them at any point keeping your research clean, concise and convenient.

[beetlemail-blog]

How To Guide: Creating Subscriptions

At Beetle we want to make your campaign research and market analysis as easy as possible, this is why we have created search Subscriptions. 

We found that once you have searched for a particular brand or campaign, you want to know what happens with this campaigns on an ongoing basis. Subscriptions allow for you to do this and refer back to whenever you need.

Here is how you create a Subscription:

Step 1.  Start your search.

Start your search on the homepage by entering the keywords or brand you are looking to see emails about.

Step 2. Refine your search

Once you have submitted your search request you will be presented with our full range of emails, since the birth of Beetle until the present day, in reverse chronological order (newest first).

To refine the date period of your search you can alter the search parameters in the top right hand section of the page.

Here you can see if I have limited the search to only show me results from 2016. The page will then reload with only the requested results. 

Step 3. Hit the blue button. 

You’ll see under the date parameter boxes there is a blue “Subscribe to this search” button. 

Click this button and a window will pop-up asking you whether you want to save as a new Subscription or replace an existing one. Name it and save it.

Now your search is saved it will automatically update with any emails since the start date you set, check in daily to see the new content and analytics available.

Step 4. Referring back to your Subscription

On the side bar menu you will see a Subscriptions option.

Click on this and your personal Subscriptions page will open. In here you will see all of the Subscriptions you have previously set up.

Select the Subscription you want to look at and it will load on the page, with the latest emails relating to your search criteria.

Make sure you check out the Analytics tab too, so you can see all the juicy information related to your search.

Have fun Subscribing!

[beetlemail-blog]

The Beetle Architecture Part 2 - The Fetcher

This is part 2 in a series of articles about what powers Beetle behind the scenes. Last time we looked at how Beetle signs up to email campaigns. This time we will look at the next step in the process; downloading our emails.

We use Google Apps to manage the multiple email domains we use when we sign up. All email gets caught by one ‘catchall’ inbox for each domain.

The Fetcher opens IMAP connections to all inboxes, and listens for new email. Legacy systems would process email as soon as it came in. As email volume ramped up we found we needed to start processing email in batch. Once the batch size is reached, all emails in the batch are downloaded at once.

Once the raw files are downloaded, we decode the email -  text and HTML part - and give it a unique name based on when we received it and the subject line. We discard any attachments.

The downloaded files are then uploaded to Amazon S3 and the details are saved to our MySQL database. We leave the content of the emails totally untouched for this stage.

In a similar fashion to how the emails are downloaded, the emails are also marked as read in batch. We ran into problems with marking lots of emails as read very quickly when our volume started to increase. 

The Fetcher is a relatively simple application, comprising one of the more simple stages the emails must go through before being visible on the site.

In the next article we’ll look at the most complicated application in the process: The Localiser.

[beetlemail-blog]

The Beetle Architecture Part 1 - The Crawler and the Binger

We frequently get questions at Beetle about the infrastructure that processes the thousands of emails we receive every day. In the following few posts, we’ll be walking through the different steps each of our emails go through before they are made visible in your feed, and the application that makes sure you keep receiving relevant emails.

You might be wondering by now what the Crawler is.

The Crawler is the first of our 7 automated steps. it’s important to understand the emails we receive aren’t as a result of any special deals with the senders. On the contrary, we want to remain under the radar and receive the emails that any Ordinary Joe would receive. That way, we know that we’re showing you the emails that real people are receiving, at the times they’re receiving them. 

However, without any special deals with the vendors, the only way we’re going to receive any emails at all is by actually signing up to email campaigns. At the time of writing this post, the number of signups to email campaigns, marketing cycles and newsletters we’ve made stands at 383,942. 977 of them were made today. As much as I’d love to tell you we have a warehouse of tiny monkeys sat at computers signing up to email campaigns all over the web, paid in bananas and red bouncy balls, I’m afraid it’s not the case.

Our MySQL database contains a table of thousands of sites to crawl. Each of these sites is crawled occasionally by a nodejs application; the Crawler. Each page is scoured for forms that look like they might take an email address. if a form is found, the crawler creates a new signup record in our Signup table, fills in the form and submits it. Each signup record consists of a set of unique demographic data (Name, address, phone number, password etc), and its own email address. By giving each signup its own email address, we can tie each email we receive back to its source - what page we signup on, when we signed up and what details we used - based on what address the email was sent to.

We take form spamming and crawler abuse very seriously, so we make sure to limit the number of requests we make to all the sites we crawl, and we’re very conscientious about how often we crawl each site. We’ve found that by ensuring that we only crawl each site every now and again, we stay in the current email campaign cycles, and receive the most up to date follow up/marketing emails, without upsetting any company sysadmins!

Every so often, our crawler will come across a form it doesn’t know how to handle. In this case, it’s down to one of the team to sign up to the site manually. In order to do this properly, with proper demographic data, we each have a copy of a specially made chrome extension installed in our browsers that returns a new set of data for us to enter into the form. This way, even manual form submissions are relatively painless. 

The crawler has a companion, which we call The Binger. Remember our table of URLs to crawl? This table also contains search terms. It’s the Binger’s job to periodically perform searches, using the API of a well known search engine (No prizes for guessing which one!) to find new sites related to these search terms. For example, we recently wanted to add a load of university sites to the crawler. Instead of inserting the university website URLs manually, we can simply enter related keywords into the Binger, and let it find the most relevant sites. These searches are performed regularly to make sure we’re always signing up to the most popular sites relating to our search terms. 

That’s about it for our high level overview of our first step to getting you the emails that you need. 

In the next post, we’ll be covering the first stage of what happens when we actually receive an email from all those sexy email campaigns we’ve signed up to! Get ready for some mega sick acronyms. Like IMAP, S3, AWS and MD5 yo! 

Until next time, feel free to drop us an email with more specific questions at [email protected]. We’ve built an architecture we’re proud of here at Beetle, and we’re always willing to share some of the knowledge we’ve accrued along the way.

[beetlemail-blog]

We have a blog!

Hi everyone! We’re planning on keeping you all up to date with the most important things we do on here. We’ll be tagging our posts with the general topic contained within, whether we’re talking about usability, technical improvements or general updates.