Tuesday, September 23, 2014

Experimenting with open source big data applications on a personal desktop

I decided to learn OpenStack, Hadoop, Elasticsearch and related open source software products using my quad core desktop. I had to raise the RAM to 16GB and add a 1 terabyte disk.

However, after the hardware upgrade, it has been remarkably simple to experiment and learn these tools using Fedora 20 repositories.

I am writing about my experiences in a monthly column in Open Source for You and will continue to post them on my site after a month or more of the publication.

I hope learning new tools is as effective as learning new languages to keep senility away after retirement :)

Saturday, August 23, 2014

Csound and x86_64 machines

I could not understand why Sugar TamTam activities wouldn't play any sound on x86_64 even though the problem seemed resolved! It took a while to realize that the problem was with x86_64 architecture. I could get the sound to play on an i686 virtual machine.

Thanks to realtime Csound programming using Python, I could trace the fault. The issue is illustrated by the following code in csoundSession.py:

        if csnd.csoundGetSizeOfMYFLT() == 8:
            pfields = csnd.doubleArray(n)
        else:
            pfields = csnd.floatArray(n)
The csound client code in Python and C used by tamtam activities assumes a float array.

The following line in csound.spec from rpm source makes the issue pretty clear:

# Csound is really dumb about 64-bit
Rebuilding csound on x86_64 with "%define useDouble 0" even for x86_64 resolved the problem with TamTam activities.

However, it is entirely possible that this workaround may cause some other applications which use csound to fail on x86_64.

Wednesday, August 13, 2014

OpenStack - Virtual Machine stuck in task state 'powering-on'

Today for the second time I had to search for the solution in http://dachary.org/?p=2939. Last time, I seemed to have selected the right keywords but it took much more effort today.

I just needed to reset task_state in instances table of the nova database:
mysql -e "update instances set task_state = NULL, \
           where deleted = 0 and hostname = 'master'" nova
The problem may be caused by error in nova compute starting up as I use virtual machine manager as well.

Virtual machine manager creates images in /var/lib/libvirt/images which are readable by root only.

For some reason, nova-compute looks in this directory as well and gets errors like:
Stderr: "qemu-img: Could not open '/var/lib/libvirt/images/fedora20.qcow2': Permission denied\n"
I manually change the permissions to readable by all to overcome the problem.

Sunday, June 15, 2014

Running TamTam Activities on Fedora 20

While preparing an old machine to give to the children of our helper, I had to struggle a bit to get TamTam activities to run. I was using the Sugar environment on Fedora 20. The version of TamTam activities in the Fedora repository was old and did not work.

The solutions were as follows:
  • The tamtamorc.cfd file in tamtam/common/Resources/ needs an additional comma in line 62:

    $ diff tamtamorc.csd tamtamorc.csd.orig
    62c62
    < aindex, atrans, ifreq, iphase, itable, itabdur xin
    ---
    > aindex, atrans, ifreq, iphase itable, itabdur xin
  • On the x86_64 systems, in common/Util/Clooper, I needed to run make to make sure that aclient.so is built for the installed version of csound.

Wednesday, February 5, 2014

Getting started with Erlang Webmachine on Fedora 20

The problems I am encountering in learning erlang on Fedora 20 is helping me understand erlang better :)

I had to install erlang-ibrowse and erlang-meck even for starting the skeleton application. They should probably be included in the dependencies of erlang-webmachine rpm.

The template for creating the skeleton application is included in
/usr/lib64/erlang/lib/webmachine-1.10.1/priv/templates.

To create a new skeleton app:
$ cd /usr/lib64/erlang/lib/webmachine-1.10.1/priv
$ rebar create template=wmskel prefix= appid=

$ cd
$ rebar compile

The wmskel.template needed to be modified to comment out the following lines related to rebar:
%{file, "{{webmachine}}/rebar", "{{prefix}}/rebar"}.
%{chmod, 8#744, "{{prefix}}/rebar"}.

The version number in rebar.config also needed to be changed from 1.9.* to 1.10.*

However, execution of start.sh fails with the error:
 init terminating in do_boot ()

The issue was that in .erl, the following statement fails:
   ensure_started(mochiweb),

It appears that some of the dependencies of mochiweb are not implicitly started. I am not sure if I am missing something, this is a fedora specific issue or a current mochiweb version issue. A work around was to modify the start function in src/.erl adding the lines in bold:

start() ->
    ensure_started(inets),
    ensure_started(crypto),
    ensure_started(asn1),
    ensure_started(public_key),
    ensure_started(ssl),
    ensure_started(xmerl),
    ensure_started(compiler),
    ensure_started(syntax_tools),

    ensure_started(mochiweb),
    application:set_env(webmachine, webmachine_logger_module,
                        webmachine_logger),
    ensure_started(webmachine),
    application:start(webmachine_demo).

For diagnosing, modifying ensure_started function in the same file helped.

ensure_started(App) ->
    case application:start(App) of
        ok ->
            ok;
        {error, {already_started, App}} ->
            ok;
        X -> io:format("App failed ~w ~w~n",[App, X])
    end.


Now,
$ rebar compile
$ ./start.sh



http://localhost:8000  should show "Hello, new world".

It may be preferable to modify the start function in templates/src/wmskel.erl.

Thursday, January 16, 2014

Getting started with Erlang Mochiweb on Fedora 20

It is simple to use this tutorial for creating a minimal Erlang Mochiweb application. Clone the git repository and run make twice.

However, trying to do the same on Fedora 20 using the packages erlang-mochiweb and erlang-rebar wasn't so simple.

While the templates are included in the /usr/lib64/erlang/lib/mochiweb-2.4.2/support directory, it is missing the Makefile. We need to execute the rebar commands instead.

Several lines need to be commented in support/templates/mochiwebapp.template. These correspond to files not included in the erlang-mochiweb rpm package -
%%{file, "../../.gitignore", "{{dest}}/.gitignore"}.
%%{file, "../../Makefile", "{{dest}}/Makefile"}.
%%{file, "../../rebar", "{{dest}}/rebar"}.
%%{chmod, 8#755, "{{dest}}/rebar"}.
The script start-dev.sh fails. Solution is to replace '\\' by '\'. The script becomes:
exec erl -pa ebin edit deps/*/ebin -boot start_sasl \
    -sname {{appid}}_dev \
    -s {{appid}} \
    -s reloader
Now,
$ cd /usr/lib64/erlang/lib/mochiweb- 2.4.2/
$ rebar create template=mochiwebapp dest= appid=
$ cd
$ rebar compile
$ ./start-dev.sh

If successful, browsing localhost:8080 shows the success page.

Friday, June 28, 2013

Relying more on Desktop Client for Emails

After switching to Akregator from Google Reader and learning to live with a little inconvenience, I was struck by a comment about the danger of email accounts being broken into by the criminal elements.

I now have setup an imap account and am using Kmail. Contents of any folder/label which need not available for easy access online, I plan to move to a local folder. I will miss out on the power of Google search, but I think, along with consuming less energy, learning to live with some technological inconveniences is the preferred/imperative option.

Update - Came across an interesting news of excessive reliance of technology -  When technology fails a news anchor, there are no words.