Drastic Actions is Drastic Actions @drasticactionsa - Tumblr Blog

VLC for WinRT: “Partial hardware decoding”

VLC pushed out a new update for Windows Phone which enables “Partial Hardware Decoding”. Now, this is a good thing. It enables all phones to be able to render out high quality video to nearly all devices, low to high end. The thing is, it’s not VLC doing it.

As many people know, at the time of this writing, Libvlc does not support hardware acceleration for ARM devices (So, Windows Phone and Windows RT). This is not ideal, but verses not having a release at all, it’s better than nothing. At least you know that VLC is actually rendering video. There is work being done to correct this, but, of course, it takes time. And apparently, with enough people on the Internet bitching about it, something had to be done.

As a stop gap, VLC for WinRT is now using MediaElement for video playback. If the video you picked does not support MediaElement, then it falls back to Libvlc. Let’s look at some code:

(InitializePlayback: MediaPlaybackViewModel.cs)

// First set the player engine // For videos AND music, we have to try first with Microsoft own player // Then we register to Failed callback. If it doesn't work, we set ForceVlcLib to true if (UseVlcLib) _playerEngine = PlayerEngine.VLC; else { var path = ""; if (!string.IsNullOrEmpty(media.Path)) path = Path.GetExtension(media.Path); else if (media.File != null) path = media.File.FileType; _playerEngine = VLCFileExtensions.MFSupported.Contains(path.ToLower()) ? PlayerEngine.MediaFoundation : PlayerEngine.VLC; }

Now on Windows Phone, we were already using the default Microsoft frameworks for audio. If the file did not support it (Ex. FLAC) then it would use Libvlc, but it would not play in the background. Now, I’m somewhat forgiving over why this had to be done. The Background Audio API on Windows Phone is, quite frankly, a royal pain in the ass to implement, and currently does not mesh well with Libvlc. It’s possible, but not practical. For video, I’m less forgiving.

To me, users are using VLC because they expect VLC to be, you know, running the file. They want the flexibility the library brings. That’s why so many people are excited about it being on the platform. But technically, right now, it’s not doing that. The built in frameworks are. If it works in the built in video player, it works here too. Exactly the same. Sure, people will be able to watch HD on their devices right now, but they could do that in other video apps on the platform. And now the video they watch will be acting exactly the same as it would in those other apps, just with a different UI... so to me, right now, VLC for WinRT is not nearly as special as it once was.

Now, of course, this stuff going to be changed out when actual hardware acceleration is ready, but I feel changing this right now to use media element is a waste of development time. I would argue (many times, as I did in the VLC ports chat room), that instead of taking the time to implement these services, other bugs and issues could have been addressed that are more important. The “Universal” UI is still a bitch to maintain, and it’s still being actively changed. The view models are a mishmash of operations and there is no separation between where they should go. The IValueConverters contain tons of business logic that should not belong there.

But that’s me.

#vlc #vlc-winrt #windows #windows-phone #winrt #c-sharp

VLC for WinRT: The Database

VLC for WinRT is a behemoth of a program. Now only does it have to handle managing your video collection (including things like separating out your normal videos from your TV shows), but also your music collection. It’s hard: Metadata is a pain in the ass to keep track of, especially when removing just one file may mean having to re-index the whole collection again. Most users, as far as I know, face tons of crashes because of how this stuff is currently handled. So let’s go into what’s going on.

When VLC for WinRT was originally released as a Windows 8 (Note, not 8.1. Specifically 8.0) app, its database structure was, frankly speaking, insane. It originally housed all of your videos and music into one large JSON file. Now, some developers may read that and just start cringing. For those who don't know who may be reading this for some reason: JSON (JavaScript Object Notation) is commonly used to pass data between a server and a client (IE: You). It’s easy to parse and turn into objects, so manipulating it whatever environment you’re in is trivially simple. For example (from Wikipedia):

{ "firstName": "John", "lastName": "Smith", "isAlive": true, "age": 25, "height_cm": 167.6, "address": { "streetAddress": "21 2nd Street", "city": "New York", "state": "NY", "postalCode": "10021-3100" }, "phoneNumbers": [ { "type": "home", "number": "212 555-1234" }, { "type": "office", "number": "646 555-4567" } ], "children": [], "spouse": null }

Simple right? Now imagine your entire music collection encoded like that, being loaded into memory every single time you started VLC. Sure, if you had a handful of albums or videos it was fine. But it’s not even close to being scalable! Start loading up the albums and your app will start to choke. One of the first major things I did for the project was introduce Sqlite (A SQL Database engine).

Unfortunately, at the time, there was no ORM (Object-relational mapping, like EntityFramework) solution for Windows Store apps. So I had to try and roll my own kind of solution.

using System; using System.Collections.Generic; using System.Text; namespace VLC_WINRT_APP.DataRepository { interface IDataRepository { void Initialize(); void Drop(); } }

using System.Collections.ObjectModel; using System.IO; using System.Linq; using System.Threading.Tasks; using SQLite; using VLC_WINRT_APP.Model.Music; namespace VLC_WINRT_APP.DataRepository { public class ArtistDataRepository : IDataRepository { private static readonly string _dbPath = Path.Combine( Windows.Storage.ApplicationData.Current.LocalFolder.Path, "mediavlc.sqlite"); private SQLiteConnection connection; public ArtistDataRepository() { Initialize(); } public void Initialize() { var db = new SQLiteConnection(_dbPath); connection = db; connection.CreateTable<ArtistItem>(); } public void Drop() { using (var db = new SQLite.SQLiteConnection(_dbPath)) { db.DropTable<ArtistItem>(); } } public async Task<ObservableCollection<ArtistItem>> Load() { var connection = new SQLiteAsyncConnection(_dbPath); return new ObservableCollection<ArtistItem>(await connection.Table<ArtistItem>().ToListAsync()); } public ArtistItem LoadViaArtistName(string artistName) { var query = connection.Table<ArtistItem>().Where(x => x.Name.Equals(artistName)); return query.FirstOrDefault(); } public Task Update(ArtistItem artist) { var connection = new SQLiteAsyncConnection(_dbPath); return connection.UpdateAsync(artist); } public Task Add(ArtistItem artist) { var connection = new SQLiteAsyncConnection(_dbPath); return connection.InsertAsync(artist); } public async Task<ArtistItem> LoadArtist(int artistId) { var connection = new SQLiteAsyncConnection(_dbPath); var query = connection.Table<ArtistItem>().Where(x => x.Id.Equals(artistId)); var result = await query.ToListAsync(); return result.FirstOrDefault(); } public Task Remove(ArtistItem artist) { var connection = new SQLiteAsyncConnection(_dbPath); return connection.DeleteAsync(artist); } } }

This is an example of the foundation of the app and how it stores data. Some of you may have started to spot the problems with this approach, but it was all I could think of at the time: A simple interface, just initialize and drop. Sqlite does not handle foreign keys in the Windows Store version I had, and there was no other addon for it for Windows Store apps that could actually handle them, so we just don't use them. From a schema standpoint, it’s nuts. We can’t even cascade delete! We just hope our code does not break when deleting them manually. But it’s the best we could come up with given the time restraints we had at the time.

Remember, the app was out. The press was talking about it. People were (trying) to use it. Sometimes you go for the quick solution instead of the right one because you need to fix it NOW, even though you know it’s not the ideal way it should be done. This is the paradox we all face in software development, and here’s another good example of it. I had work obligations to handle, so I had to drop out of the project for a few months after I gave them what I had. They would enhance it and the album parsing logic to try and make it faster.

In a perfect world, EntityFramework 7 will be released soon with fully working Windows Store support (It was in the beta before they dropped it to finish Asp.Net vNext support. Which is more important for my main line of work, but it would be nice here too ;) ). We could use that to handle all our ORM needs. But for now, it's the best we could do...

Soon we will move to the business access layor (or BAL), the view models, and how they use this data.

#vlc #csharp #winrt #windows-phone #windows

VLC for WinRT: a “true, Universal App”

One of the big points Thomas made in the new VLC for WinRT release was it becoming a “true, Universal App”

As the frontend guy, my job this last month was to completely merge the Windows and Windows Phone projects, and move quite everything in the Shared project. Today we’re sharing ~80% of our XAML code and >95% of the C# code, and >this will keep improving during the next couple of months.

The tech press picked up on those stats. But it’s quite clear to me that they don’t actually know what that means. Seeing “move quite everything in the Shared project” is really cool, but it’s not without issues of its own (Especially with numbers that big). But let’s go into why, and how this approach could potentially make more sense with Windows 10.

At Build 2014, Microsoft announced WinRT (The framework used for building “Modern” Windows Store applications) for Windows Phone 8.1, as well as concept of “Universal apps”. When you either enable an existing app to be a universal app, or create a new one, you get three projects: The Windows project, the Windows Phone project, and a shared project. Here you can place platform specific XAML or view code (AKA, what you see on screen) in the platform specific projects, and shared code (Like, for example, view models, commands, or shared views like user controls) in the shared solution. If you’re good and know how to properly follow the MVVM (Model-View-View-Model) pattern, it’s relatively easy to leverage the code in the shared project into the platform specific views in the other projects. So once you finish working on one project (be it Windows 8 or Windows Phone), it’s just a matter, more or less, of building the views and remaining platform specific logic for the other.

This is the (closed) solution for VLC for WinRT. The important bits for us are the Windows, Windows Phone, and Shared projects.

In the shared solution, we see tons of views. Let’s look AlbumCollectionBase. This is used to show you your list of Albums.

Here is the XAML preview. The design window is set to 300px, but when placed in our views it scales to whatever the size is set to it (Normally, it’s set to match the total width of the device size). Now, remember that this code is in the shared solution. This gets run in both the Windows Phone and Windows 8 apps.

Now, the bit I’m interested in is the list view, which actually shows you the albums. Let’s look at it.

And more importantly, the itemswrapgrid, used to define the constraints on the items themselves in the listview:

<ListView.ItemsPanel> <ItemsPanelTemplate> <ItemsWrapGrid SizeChanged="AlbumsWrapGrid_SizeChanged" Orientation="Horizontal" Margin="0,0,0,60" /> </ItemsPanelTemplate> </ListView.ItemsPanel>

“SizeChanged="AlbumsWrapGrid_SizeChanged"”. This means that when the size on the ItemsWrapGrid changes (What the albums themselves are housed in), it fires this command in the code behind on the view. Let’s look at that command.

private void AlbumsWrapGrid_SizeChanged(object sender, SizeChangedEventArgs e) { TemplateSizer.ComputeAlbums(sender as ItemsWrapGrid, TemplateSize.Compact, this.ActualWidth); }

So it sends the width of the grid, as well as the grid itself, to another helper function in order to set how many items you see.

public static void ComputeAlbums(ItemsWrapGrid wrapGrid, TemplateSize size = TemplateSize.Compact, double? width = null) { if (width == null) width = Window.Current.Bounds.Width; var splitScreen = (size == TemplateSize.Compact) ? 3 : 2; #if WINDOWS_PHONE_APP if(width == 400) size = TemplateSize.Normal; if (!DisplayHelper.IsPortrait()) splitScreen = 5; var itemWidth = (width.Value / splitScreen); var itemHeight = itemWidth + 40; wrapGrid.ItemWidth = itemWidth; wrapGrid.ItemHeight = itemHeight; #else if (width > tileSize*3) { wrapGrid.ItemWidth = tileSize; wrapGrid.ItemHeight = tileSize + contentGridWindows; } else { wrapGrid.ItemWidth = (width.Value / splitScreen); wrapGrid.ItemHeight = wrapGrid.ItemWidth + contentGridWindows; } #endif }

“#if WINDOWS_PHONE_APP” is a conditional statement that only gets fired when the app is being run on Windows Phone. Likewise if you use “#if WINDOWS _APP”. So we are getting conditional code for each platform. So we are in the shared solution doing platform specific view logic… which seems to defeat the purpose of putting it here. This happens all over the place in the shared project view files. Granted, it’s mostly trivial changes (Changing the thickness of lines, or margins, what have you), but these little statements can add up, making it very hard to debug and make changes without screwing up the other project.

Let's say we have an issue with one of these views, or decided we want it to behave differenty or have platform specific commands be shown to the user. By having it layed out like this, it's much more difficult to do then it needs to be.

What we really want is more platform independent XAML. It makes sense to build user controls that work across Windows Phone and Windows 8 and putting them into the shared solution. But right now, you can still end up with too many platform specific If-Else statements that’ll just make it a pain in the ass to maintain.

But with Windows 10, Microsoft seems to be making moves to make this sort of thing possible “and easier to do”

They’ve been talking a lot “Adaptive UX” and having one “experience” work across multiple platforms. By having better handling of device profiles and layouts, us developers hopefully won’t have to write these elaborate views to try and get everything to scale ourselves. We can write just one layout and have it scale across phones, tablets, and desktops. That’s what VLC tried to do here, but just a bit too soon. It’s too fragile for my taste to have THIS much code placed in the shared project, with the amount of conditional statements needed to make it work. But seeing where Microsoft is heading, porting our current stack to the true Windows 10 APIs might actually be easier to do, seeing that most of the code is already in the right locations. We would just have to get rid of the conditionals.

We’ll see what Build brings.

#vlc #winrt #csharp #windows

VLC for WinRT: Album Art and Artist Headers

Before I begin: I am a contributor to the VLC for WinRT project. My opinions are my own and not of the VLC Project. Again, I just send patches. That's all.

A big issue with VLC for WinRT is how album and artist information is handled. You would expect album art to just be pulled from the music file itself. Hell, there is an API in Windows just for doing this!

using (var thumbnail = await _file.GetThumbnailAsync(ThumbnailMode.MusicView, 100)) { if (thumbnail != null && thumbnail.Type == ThumbnailType.Image) { var bitmap = new BitmapImage(); bitmap.SetSource(thumbnail); image.Source = bitmap; } }

But this won't work. Windows Phone 8.1 will always return null. Why? No one knows. It's a bug within the framework.

EDIT: Turns out I'm wrong on that one. It's not supported. Thanks to Rudy Huyn for pointing that out to me. But still, it's a stupid limitation.

Now, we could work around this by using TagLib. A few months ago, I tried to add TagLib# to the VLC WinRT project and leverage it to get our album art. It worked, but when I submitted the page, I got yelled at.

You see, LibVLC already has Taglib bundled in, so adding an outside library that already had this functionality was not necessary. The only problem was that it was not exposed in our framework. I don't know how to do that, and no one else actually has yet. So right now, any time you start indexing music in VLC, it pulls down album art from Deezer and LastFM.

Artist information is pulled down when you try and access an artist from the music view. Currently it does not save the biography in the database, so it always pings LastFM (I so want to change that, there is zero reason for this), but the artist header is pulled down and saved)

While this is not ideal, especially if you're on a mobile connection, it works. Usually it gets the correct album art or artist image, and everyone is happy. Usually.

That's the wrong Polaris. Now, I'm going to be fair to LastFM here; Polaris is a pretty generic name for a band, and the one I was actually interested in is not on LastFM. So that's understandable.

Shockingly, LastFM does not have album art for a Bad Religion bootleg. It also pulled in art for the "New America" single, rather than the album itself. Now, they are named the same, so I can understand the confusion. But it is weird it would go for the single first, rather than the album itself.

Before we had no way of changing them. So if you got the wrong art, too bad. You had to deal with it. So I went and added the option.

But I ran into issues. If you didn't have an artist header or album art for that album, it referenced a generic "no cover" image. So saving a new one in the database was fine, since one did not exist yet. But if you did have one, and I tried to overwrite it, I would get "Access Denied" errors, saying the file was being accessed.

Now, the image was loaded and in memory. How was it still being accessed? This usually pops up when a stream is not closed, but our album art and artist headers were being referenced as URIs in our XAML. What was happening was that if you binded or referenced an image via a string or URI in XAML, it would never close the stream. So if you ever tried to access it (or in my case, delete it), it would fail.

The only solution I could think of (which was and is no way ideal, but it was only thing I could think of) was replacing the binding with a bitmap image instead of a string. This way, we could load the image and properly close the stream ourselves.

public static async Task LoadImageToMemory(ArtistItem item) { try { var fileName = string.Format("{0}.jpg", item.Id); var albumPic = await ApplicationData.Current.LocalFolder.GetFolderAsync("artistPic"); var file = await albumPic.GetFileAsync(fileName); var stream = await file.OpenAsync(FileAccessMode.Read); var image = new BitmapImage(); image.SetSource(stream); stream.Dispose(); item.ArtistImage = image; return; } catch (Exception ex) { LogHelper.Log("Error getting artist picture : " + item.Name); } // Failed to get the artist image or no cover image. So go online and check // for a new one. await item.LoadPicture(); }

Now, this does work. If we fail to load an image, the try catch logs it and we try to get one from LastFM. But it's shit right? You should not use a try catch in this way. But it was a fast solution.

After I submitted the patch, Thomas Nigro (The main frontend dev on the project) made some changes that make a little bit more sense.

public static async Task LoadImageToMemory(AlbumItem item) { bool fileExists = false; try { var fileName = string.Format("{0}.jpg", item.Id); fileExists = await ApplicationData.Current.LocalFolder.ContainsFolderAsync("albumPic"); if (fileExists) { var albumPic = await ApplicationData.Current.LocalFolder.GetFolderAsync("albumPic"); fileExists = await albumPic.ContainsFileAsync(fileName); if (fileExists) { Debug.WriteLine("Opening file albumPic " + item.Id); var file = await albumPic.GetFileAsync(fileName); var stream = await file.OpenAsync(FileAccessMode.Read); var image = new BitmapImage(); image.SetSource(stream); stream.Dispose(); item.AlbumImage = image; } } } catch (Exception) { LogHelper.Log("Error getting album picture : " + item.Name); } if (!fileExists) await item.LoadPicture(); }

It's still not ideal, but it makes a little more sense. Check and see if the file exists first, and if it does, load it. Otherwise check online and try and get one (Which is what "LoadPicture" does, we should really change the name of that). I should have really thought of that in the first place. That fixes the "Use a try catch in a shitty way" issue but it's still not totally right. The best solution would probably be to actually mark in the database if an album or artist has online or local artwork. If it does, use it, otherwise use the generic cover. But that requires database changes and since we don't have an ORM that actually works well on Windows Phone yet (Maybe when EF7 is stable we could switch. One can dream), it's a real pain to change. But hey.

It does work. So that's something. At least I don't have to look at random bands in my Artist list now. Small steps.

#vlc #winrt #csharp

Trending Blogs

Recently Viewed Blogs

Drastic Actions is Drastic Actions