Metinsel Maceralar

Sıradışı bir işi olan sıradan biriyim.

[onurtugcu]

Finally Python

After a year of having to use python3, I decided to write an opinionated piece on how to use python3:

First, install and update package installation and environment tools. Replace python3.7 with python3.6 if 3.7 is not available on your platform.

python3.7 -m pip install -U pip wheel setuptools venv --user

Second, create a project directory.

mkdir fabulous cd fabulous

Third, create a virtual environment inside the project directory and activate it.

python3.7 -m venv venv source venv/bin/activate

You can use deactivate to exit this mode any time. Now you are able to use python and pip programs directly.

If you need, install dependency management, syntax highlighter. I just install them on almost all environments.

pip install autopep8 mypy pipdeptree pip-autoremove pip freeze > requirements.txt

You are now ready to develop your own programs. You can see packages you've installed with pipdeptree. If you want to remove a package, you can do so with pip-autoremove <package name> which will also remove all unused dependencies.

If you or anyone who joins your project want to use an IDE like vs code, it should automagically detect your autopep8 syntax formatter and your mypy linter.

Note that if you are programming on a constrained environment such as a raspberry pi, odroid or any other armv7 system with low memory, installing packages like scipy or numpy will take a very long time. In that case, if you want to speed things up, try using your operationg system's package manager and avoid the virtual environment.

[onurtugcu]

So I was connecting Qt sockets to callbacks the other day. Turns out QSignalMapper is finally deprecated and we can use lambdas instead.

void MainWindow::on_pushButton_clicked() { if(server != 0) { delete server; } server = new QTcpServer(this); server->listen(QHostAddress::AnyIPv4, 4243); connect(server, &QTcpServer::newConnection, [this](){ while(this->server->hasPendingConnections()) { QTcpSocket* socket = this->server->nextPendingConnection(); connect(socket, &QTcpSocket::readyRead, [socket](){ QByteArray request = socket->readAll(); socket->write(request); }); connect(socket, &QAbstractSocket::disconnected, socket, &QObject::deleteLater); } }); ui->pushButton->setText("Server Started on 4243, Click to restart"); }

[onurtugcu]

SQL Art

This little gist lets you group rows by a repeating pilot row.

select         i         , c         , g         , gi         , sum(1) over (partition by g) gn from ( select         i         , c         , g         , row_number() over(partition by g order by i) gi from ( select          i         , c         , sum(case c when 'a' then 1 else 0 end) over (order by i) g from (         select 1 i, 'a' c union all         select 2, 'b' union all         select 3, 'c' union all         select 4, 'a' union all         select 5, 'b' union all         select 6, 'a' union all         select 7, 'b' union all         select 8, 'c' union all         select 9, 'd'  ) t ) t ) t order by i

[onurtugcu]

python3 for frustrated rubyists

I will be logging my first attempt to make use of python because rubyists with actual real jobs just couldn't keep up with infinite numbers of undergraduates constantly contributing packages to python no matter how much it sucks.

To get started:

sudo apt install python3 python3-pip

It is important to create a virtual environment for your project. It helps isolate your project from the rest of your OS so if some system-wide python package changes, it (hopefully) won't break your project. Now go to your project graveyard directory and type:

python3 -m venv newproject cd newproject . bin/activate

And now you think you can install packages into your virtual environment? You are wrong. Virtual environment will be equipped with an old pip3 which won't let you install anything. You first need to upgrade pip

pip3 install --upgrade pip

Now you can install packages

pip3 install mechanize

Or so you think. Until you see mechanize only works on python 2.x ha-ha fooled you. Go Google for alternatives.

pip3 install mechanicalsoup

Worked fine. Why am I installing scraping tools? well, you know you want to parse websites. Admit it. Now let's dump the requirements to a file so we can move the project to another machine and it will still work.

pip3 freeze > requirements.txt

Now we can move the project to another environment and recover the requirements

pip3 install -r requirements.txt

So that was python. You encounter two stack traces just trying to get started. Compare it to ruby:

rvm install ruby-head gem install bundler mkdir newproject cd newproject echo "https://rubygems.org" >> Gemfile echo "gem \"mechanize\"" >> Gemfile bundle

And now you have a Gemfile.lock with your frozen environment in it. No stack traces.

[onurtugcu]

Vim Typescript Autocomplete

In this post I will be logging how I got autocomplete for typescript in vim.

In short, I installed pathogen with a bunch of plugins and scoured github issues pages so you don't have to.

On debian/ubuntu, you need vim-nox package in order to satisfy the lua dependency of neocomplete.vim.

First install pathogen (for everything else)

mkdir -p ~/.vim/autoload ~/.vim/bundle && \ curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim

Then install vimproc for tsuquyomi

mkdir -p ~/.vim/bundle git clone https://github.com/Shougo/vimproc.vim.git ~/.vim/bundle/vimproc.vim pushd ~/.vim/bundle/vimproc.vim make popd

Now install tsuquyomi

git clone https://github.com/Quramy/tsuquyomi.git ~/.vim/bundle/tsuquyomi

Install neocomplete

git clone https://github.com/Shougo/neocomplete.vim ~/.vim/bundle/neocomplete

Now set them up in ~/.vimrc (tsuquyomi needs to find tsserver)

" make pathogen work execute pathogen#infect() " typescript omnicomplete let g:tsuquyomi_use_dev_node_module=2 let g:tsuquyomi_tsserver_path='/home/nurettin/npm-global/bin/tsserver' autocmd FileType typescript set omnifunc=tsuquyomi#complete " neocomplete start with a dot let g:neocomplete#enable_at_startup = 1 if !exists('g:neocomplete#force_omni_input_patterns') let g:neocomplete#force_omni_input_patterns = {} endif let g:neocomplete#force_omni_input_patterns.typescript = '[^. *\t]\.\w*\|\h\w*::' "neocomplete tab complete inoremap <expr><tab> pumvisible() ? "\<c-n>" : \ <sid>check_back_space() ? "\<tab>" : \ neocomplete#start_manual_complete() function! s:check_back_space() "{{{ let col = col('.') - 1 return !col || getline('.')[col - 1] =~ '\s' endfunction"}}} filetype plugin on

Hopefully that will get you started.

Addendum:

After a bit of usage I started getting an error message:Cannot compile external modules unless the --module flag is provided

In order to avoid tsserver --module flag errors, you need to create a tsconfig.json file that specifies which files are modules.

{ "compilerOptions": { "module": "commonjs" }, "files": ["MyModule1.ts"] }

[onurtugcu]

rvm + vim + zsh

Fixing ruby environment issue within vim:

:!ruby --version mac ruby 2.0.0-dev sudo mv /etc/zshenv to /etc/zshrc

Start a new terminal for this to take affect. Now you will be able to use your rvm environment within vim.

:!ruby --version rvm ruby 2.2.3

[onurtugcu]

Using Delphi XE7+ DLLs from Delphi7

The only reasonable method I've found so far:

Isolate XE7 DLLs so that they are self-sufficient and you can pass data via WideStrings. (TClientDataSet helps with that)

use MidasLib inside the DLL. This will bundle MidasLib.dcu into your DLL project, which will prevent the DLL hell between Delphi7 and XE7 versions during deployment.

Use WideString on both sides of code. On Windows, WideString is implemented via WinAPI's BSTR and it is compatible between implementations.

Use ShareMem on both sides of code. You need this in order to load and free the DLL handle without getting access violations.

[onurtugcu]

Protect your linux against shellshock vulnerability

To test if you have the vulnerability:

env VAR='() { :;}; echo Bash is vulnerable!' bash -c "echo Bash Test"

To update your debian based system:

sudo apt-get update && sudo apt-get install --only-upgrade bash

To update your redhat based system:

sudo yum update bash

[onurtugcu]

shellshock bash Hatasına Karşı Önlem Alın

shellshock, posix dosya izin altyapısında çalışan bash skriptlerinde kullanıcının izinlerini yükseltmesine izin veren bir yazılım hatasıdır.

shellshock, otoritelerce OpenSSH heartbeat hatasından daha ciddi sonuçlar doğurabilecek bir hata olarak görülmektedir.

shellshock özellikle internete açık ssh sunucularını ve mod-cgi modunu kullanarak bash skript çalıştıran apache sunucularını ilgilendirmektedir.

Sisteminizin shellshock'tan etkilenip etkilenmediğini anlamak için şu komutu çalıştırabilirsiniz:

env VAR='() { :;}; echo Bash is vulnerable!' bash -c "echo Bash Test"

Debian bazlı sistemlerde son güncellemeyi almak için:

sudo apt-get update && sudo apt-get install --only-upgrade bash

[onurtugcu]

Encoding (Metin Kodlama) Hakkında

Bu yazıda WINDOWS-1254, ISO-8859-9 ve UTF-8 metin kodlamaları hakkında bilgiler sunacağım ve Ruby dili ile örnekleyeceğim.

Metin kodlaması, metinlerin yazarın kullandığı programdan okurun kullandığı programa aktarılması sırasında kullanılan bir cins protokoldür.

Örneğin bir yazı programı başlatalım ve metin kodlamasına bakalım:

$ irb 2.1.2 :001 > "abc".encoding => #<Encoding:UTF-8>

Farklı kodlama yöntemiyle aynı programı yeniden başlatıyorum:

$ irb -Eiso-8859-9 2.1.2 :001 > "abc".encoding => #<Encoding:ISO-8859-9>

Yani metnin içeriği yazarın kullandığı programın varsayılan kodlamasına ya da metni kaydederken seçtiği kodlamaya bağlıdır.

Kodlamalar arasındaki farkları metin karakterlerinin sayısal değerlerini ekrana yazdırarak görebiliriz:

$ irb 2.1.2 :001 > "şey".encode("iso-8859-9") => "\xFEey"

Burada gördüğümüz çıktı, UTF-8 olarak kodlanmış bir metnin ISO-8859-9 kodlamaya hatasız olarak çevrilmesi sonucunda özel karakterlerin 16 tabanındaki değerleridir.

$ irb -Eiso-8859-9 2.1.2 :001 > "şey".encode("utf-8") => "\u00C5\u009Fey"

Bu çıktı ise ISO-8859-9 olarak kodlanmış bir metnin UTF-8 kodlamasına hatasız çevrimi sonucu elde edilen 16 tabanındaki değerleri gösterir. Görüldüğü üzere ş harfinin ekranda gösterilebilmesi için UTF-8 kodlamasında iki byte bilgi tutulmaktadır.

ş harfinin UTF-8 kodlaması hakkında ek bilgi için tıklayın.

[onurtugcu]

Handling QNetworkReply Safely and Concurrently

In this post I'm going to explain a few pitfalls of using QNetworkAccessManager and how to avoid them elegantly using C++11 features.

Problem 1: QNetworkAccessManager's finished(QNetworkReply) and error(QNetworkReply) signals

The main drawback of handling replies using QNetworkAccessManager's signals is that when using the same signal concurrently for multiple replies, there is a chance of mixing them up.

Problem 2: QNetworkReply's finished() and error() signals

The drawback of handling replies using QNetworkReply's signals is that the signals do not have a QNetworkReply parameter, which means we have to manage the reply manually, probably storing it as a class member. However this would mean storing a pointer for every request and there is no way of knowing which reply the signals are called for.

There is also an issue of cleaning up memory after reply is finished or when there's a network error.

Lambdas: Solution to all problems

Using a hybrid solution of passing C++11 lambdas as slots and binding the reply as a parameter, we can avoid concurrency problems as well as handle memory cleanup.

Here's an example of posting some data to an http server safely and concurrently.

QUrl url("http://url"); QNetworkRequest request(url); QUrlQuery query; query.addQueryItem("key", "value"); request.setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded"); QNetworkReply* postReply= net->post(request, query.toString(QUrl::FullyEncoded).toUtf8()); void(QNetworkReply::*error_fun)(QNetworkReply::NetworkError)= &QNetworkReply::error; connect(postReply, error_fun, std::bind([=](QNetworkReply* reply, QNetworkReply::NetworkError){ emit subscribeError(reply->errorString()); reply->deleteLater(); }, postReply, std::placeholders::_1)); connect(postReply, &QNetworkReply::finished, this, std::bind([=](QNetworkReply* reply){ reply->deleteLater(); }, postReply));

[onurtugcu]

Worms Reloaded Steam Running Slow Issue Solution

The slowdown problem comes up when you install Worms Reloaded™ using steam.

TL;DR

Edit Default.cfg located in the game's folder and add:

/LOG:ERROR

The problem is caused by a bad default configuration which leaves Worms Reloaded™ at a debug state which is programmed to write logging information to a file. This log writing, if done every time a frame of the game is rendered, causes excessive disk usage and slows down the game.

[onurtugcu]

Troubleshooting Steam on Debian Sid 64bit

Note: Following info is potentially dangerous and you might end up removing your entire graphical subsystem and having to install it again.

steam depends on packages from the i386 architecture. Enable i386 repositories if you haven't done so:

sudo dpkg --add-architecture i386 sudo apt-get update

Sid is a quickly evolvin distro and steam might encounter errors when installing its dependencies. In order to solve them, try using a more intelligent apt package manager called aptitude.

When you encounter dependency errors, try installing requirements like this:

sudo apt-get update sudo aptitude install libgl1-mesa-dri:i386 libgl1-mesa-glx:i386 libc6:i386

Be careful! aptitude will probably give you the choice to remove your entire X server. **If it does, Say no! **

The following actions will resolve these dependencies: Remove the following packages: <220 packages listed> Accept this solution? [Y/n/q/?]

If you get such a big software removal question, REJECT IT by pressing n.

Press n until aptitude asks to remove only a few packages and upgrade a few packages. Accept this by pressing Y

Let aptitude do its work, then run steam again. This should fix your dependencies by updating only the necessary packages and deleting only a few old ones.

[onurtugcu]

C++11 MySQL tutorial

Today we are going to connect to a MySQL database using C++ as elegantly as possible. Of course everyone's definition for elegant differs, so I'm going to explain mine: No boilerplate classes, code reuse, exception safety.

The following function template makes it easier to use C APIs which have cleanup functions. ( note that this will be replaced by C++14's make_unique which uses parameter forwarding )

template <typename T, typename F> std::unique_ptr<T, F> gc(T* p, F f) { return std::unique_ptr<T, F>(p, f); }

And we will need some error checking:

void ensure_mysql(MYSQL* p){ if(p== 0) throw std::runtime_error(mysql_error(p)); } void ensure_query(MYSQL* p, int ret){ if(ret> 0) throw std::runtime_error(mysql_error(p)); }

And now we initialize client library and connect to the database in just a couple of lines:

auto db= gc(mysql_init(0), mysql_close); ensure_mysql(db.get()); ensure_mysql(mysql_real_connect(db.get(), "server", "username", "password", 3306, 0, CLIENT_COMPRESS));

Now we can query the database just as easily. For this query, we are going to choose to read row by row instead of storing the entire result set in memory. This is the recommended and quick way unless you are doing a lot of processing and stalling the data stream. For that case, you would rather want to use mysql_store_result.

ensure_query(mysql.get(), mysql_query(mysql.get(), "select table_name, table_type, engine from information_schema.tables")); auto result= gc(mysql_use_result(mysql.get()), mysql_free_result); while(MYSQL_ROW row= mysql_fetch_row(result.get())) { // access row[0], row[1] ... and convert data types here }

Here's the entire test case: (main.cpp)

#include <memory> #include <unordered_map> #include <iostream> #include <mysql/mysql.h> template <typename T, typename F> std::unique_ptr<T, F> gc(T* p, F f) { return std::unique_ptr<T, F>(p, f); } void ensure_mysql(MYSQL* p) { if(p== 0) throw std::runtime_error(mysql_error(p)); } void ensure_query(MYSQL* p, int ret) { if(ret> 0) throw std::runtime_error(mysql_error(p)); } int main() { try { auto mysql= gc(mysql_init(0), mysql_close); ensure_mysql(mysql.get()); ensure_mysql(mysql_real_connect(mysql.get(), "server", "username", "password", "database", 3306, 0, CLIENT_COMPRESS| CLIENT_FOUND_ROWS)); ensure_query(mysql.get(), mysql_query(mysql.get(), "select table_name, table_type, engine from information_schema.tables")); auto result= gc(mysql_use_result(mysql.get()), mysql_free_result); while(MYSQL_ROW row= mysql_fetch_row(result.get())) {} } catch(std::exception &ex) { std::cerr<< ex.what()<< '\n'; } }

Compile and run on linux:

clang++ main.cpp -l mysqlclient && ./a.out

[onurtugcu]

Steam Run Civ5 on Debian Nvidia Optimus Laptop

I'm going to log how I got Civilization 5 purchased through Steam to run on Linux.

After trying various configurations and packages, I settled with installing

$ sudo apt-get install nvidia-drivers primus bumblebee

This gave me a bumblebeed daemon and a program called optirun which can be used to start graphical programs.

You can test the bumblebeed daemon using these commands:

$ sudo apt-get install mesa-utils $ glxinfo |grep vendor server glx vendor string: SGI client glx vendor string: Mesa Project and SGI OpenGL vendor string: Intel Open Source Technology Center $ optirun glxinfo |grep vendor server glx vendor string: NVIDIA Corporation client glx vendor string: primus OpenGL vendor string: NVIDIA Corporation

Now simply run

$ optirun steam

Then install Civ5 and run it from steam.

[onurtugcu]

C++ programlarınız (hâlâ) yavaş mı çalışıyor?

Kodlama şeklinizi yeniden düşünmeniz gerekebilir. Örn:

nesne-> ozellik()-> yap();

Tarzı kodlamalar hafızada rastgele yerlere erişim sayınızı arttıracağından CPU cache ve prefetcher kullanımınızı azaltacak, programlarınızı yavaşlatacaktır.

Mümkün olduğu kadar diziler veya std::vector üzerinde döngü yapacak şekilde programlayın. Aşağıdaki örnek C++11 için:

foreach(auto &e: tum_nesne_ozellikleri) e.yap();

Bu tarz programlamada hafızada ardışık duran bölgeler üzerinde çalıştığınız için CPU Prefetcher devreye girip ihtiyacınız olan hafıza bölgelerini sizin için önden tespit edecek, onları önden getirecek ve programınızın hızında kat be kat artış sağlayacaktır.

[onurtugcu]

How to encode text in Ruby

Yes, you don't have to read long blog posts that go off to all kinds of tangents or wade through stackoverflow answers. It's really simple.

Let's assume you downloaded a piece of string from the internet

str= download_iso8859_9_string_from_internets

And you want to convert it to UTF-8 because everything should be in UTF-8 after all.

str.encoding

=> ASCII-8BIT

Now we are going to reinterpret str with a different encoding.

str.force_encoding("ISO-8859-9")

Now your ruby string and data content encodings match. Finally we can encode it in UTF-8.

str.force_encoding("ISO-8859-9").encode("UTF-8")

This will cause the string to be converted from its ISO-8859-9 encoding to UTF-8. It's that simple. Now you can use the returned UTF-8 string.