BSD Planet * BSD People

Hubertf's NetBSD Blog: FOSDEM talks/slides: pkgsrc on MirBSD, pkgin

Mon, 06 Feb 2012 13:18:00 +0000

Last weekend the Free and Open Software Developer Meeting (FOSDEM) happened. As in previous years, there was a booth manned by people from various BSD-projects, and there were also two(?) talks related to NetBSD and its related project pkgsrc:

NetBSD and MirBSD developer Benny Siegert gave a talk titled "pkgsrc on MirBSD" - see his slides! pkgsrc is a framework for packaging and building 3rd party applications from source. Besides MirBSD, it runs on many other platforms like Linux and Mac OS X.
While building from source is fine, it costs a lot of time. pkgsrc can also create binary packages, and to manage those, there is "pkgin", a binary package manager. Its developer, Emile 'iMil' Heitor introduced it in a talk - see the slides (PDF)!

Hubertf's NetBSD Blog: Automatic building of Amazon EC2 images from NetBSD

Sun, 05 Feb 2012 18:17:00 +0000

NetBSD/xen is available for some time now to work on Amazon's Xen-based Elastic Cloud Computing (EC2) platform, as previously announced in the NetBSD blog, and the NetBSD wiki has instructions on how to subscribe to Amazon, launching and rebuilding the "AMI" images.

This work is continued by Jean-Yves Migeon, who is working on build scripts for Amazon EC2, so the "AMI" images can be provided easily, with the eventual goal to include them into the NetBSD build process by Jeff Rizzo, so EC2 images can be automatically generated easily, e.g. by NetBSD's build cluster.

Hubertf's NetBSD Blog: Google Summer of Code 2012 will happen - first NetBSD directions

Sat, 04 Feb 2012 23:40:00 +0000

Google announced at Fosdem that there will be Google Sommer of Code 2012, the 8th time in a row. Ot can be expected that NetBSD will strive to participate again this year, and as such, now is a good time to

make up your mind if you want to participate, either as student or mentor
hash out details of project proposals and possible implementation details, and also see how much time you can devote as possible mentor. Also, think about choosing criteria for students and how to communicate with them on a daily basis and also in cases where problems arise (reallife on either side, student going AWOL, ...)
go over the list of project proposals, esp. the ones with the right dimension for GSoC, and suggest changes and additions of new projects
make yourself familiar with NetBSD from a user/admin viewpoint and esp. from a developer point if you intend to apply as student.
A (slightly dated?) tour through the NetBSD source tree is available for userland, libraries, and the kernel. Also of interest: a guide on NetBSD internals.
review our guidelines for applying for a project if you're an interested student. We get many really bad, dull and uninformed project proposals each year, and we wish more students would be as serious as YOU probably are (as you're already here :-).

Let's look forward to NetBSD and the Goole Summer of Code 2012, and the exciting new projects to come!

Hubertf's NetBSD Blog: NetBSD Hackathon - February 10th to 12th, 2012

Fri, 03 Feb 2012 21:27:00 +0000

Matthias 'tron' Scheler announced per mail and on the NetBSD blog ``The 16th NetBSD hackathon will be run from February 10th to February 12th. Our goal is fixing all the bugs that need fixing to get NetBSD-current ready for the creation of the NetBSD 6.0 release branch.
Everybody that has an interest in NetBSD, from developers, documentation writers, translators, to advanced users are invited to attend. To make sure that NetBSD users get the best possible experience of the new release we would like to fix as many bugs as possible. For a list of bugs and more information look at the Wiki Page under <https://wiki.netbsd.org/hackathon/> please.
If you are able to help us fixing these bugs by supplying patches or testing fixes please consider to participate. We are also in need of people to supply documentation fixes, preferably in the form of patches. Release notes and/or manual pages!
Join us on the IRC channel #netbsd-code on freenode (irc.freenode.net). Just join, have a look around and ask your questions or what work needs to be done.

We are looking forward to seeing you!''

Indeed! :-)

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Wed, 01 Feb 2012 21:48:05 +0000

This comes two days late... but anyway, I felt like posting it now instead of waiting until next Sunday/Monday.<br /><br />The activity past week focused mostly on implementing support for the <tt>require.memory</tt> test-case metadata property recently introduced into ATF. This was non-trivial due to the need to write some tricky Autoconf code to make this "slightly portable".&nbsp;Seriously: It's scary to see how hard it is to perform, in a portable manner, an operation as simple as "query the amount of physical memory"... but oh well, such are the native Unix APIs...<br /><br />I later spent the weekend on Lutok, preparing and publishing the project's first release and writing my first RPM library spec for it. This was a prerequisite for the upcoming 0.3 release of Kyua and thus deserved special attention!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-3471721892781390132?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: EuroBSDCon 2012: Warsaw, Poland

Wed, 01 Feb 2012 21:12:00 +0000

Quoting from the website:

``EuroBSDcon2012
18 - 21 October 2012, Warsaw, Poland

EuroBSDcon is the European technical conference for people working on and with BSD based operating systems and related projects. EuroBSDcon 2012 is the 11th EuroBSDcon and will take place in Poland, 18-21 October 2012 in Warsaw. EuroBSDcon is a great and unique time to learn more about the powerful BSD systems we use everyday and to connect with other developers around the world. ''

Hubertf's NetBSD Blog: IPv4 address change for {many}.NetBSD.org

Tue, 31 Jan 2012 22:14:00 +0000

spz@ announces: ``{mail, www, anoncvs, blog, wiki, releng}.NetBSD.org are changing IPv4 address from something in 204.152.190 to something in 149.20.53. Do not be alarmed. :)
There may be some glitches due to IP addresses hiding in unexpected corners; we apologize in advance for any issues caused by the renumbering.

The old addresses are going to be available at least another week.''

Background of this change that the ISP of most of the NetBSD services requires renumbering. Of course this only affects IPv4, not IPv6 :-)

Hubertf's NetBSD Blog: Latest IPfilter merged into NetBSD-current

Mon, 30 Jan 2012 22:51:00 +0000

Darren Reed is the author if IPfilter and also a NetBSD developer. IPfilter is one of the packet filters available in NetBSD, and the latest version (5.1.1) was imported into NetBSD-current by darren. Citing from his mail to tech-net, there are a few interesting changes and new features:

``To start with, the man pages for ipf(5) and ipnat(5) have been rewritten from scratch to make them easier to understand and thus easier to use the various features in IPFilter. In addition there is now an ipmon(5) that supports delivery of log messages to different destinations - including generating SNMP traps messages.

There are a few new actions that can be used with ipnat.conf. The one that will be of most interest to people is "rewrite" which supports translation of both the source and destination address with a single rule. Use of an rdr/map combination is no longer required. There are also some others that are more experimental. One of those is a "divert" action that takes a packet and puts an IP + UDP header on the front, allowing "raw packets" to be delivered to any socket. Similarly, replies from that socket have the relevant header data removed.

There are a few extras for ipf.conf, most notably it now allows for defining limits on how many different hosts/networks can have a state entry in the state table for each rule. IPFilter 5.1.1 also supports specifying a filter rule group for the filtering of ICMP packets that match an entry in the state table. Additionally, there is a new rule - "decapsulate". This has been designed to allow filtering on "inner headers" of packets that have been encapsulated in clear text. It will, for example, allow filtering on IPv4 headers inside of IPv6 packets (or vice versa.)

It is no longer required to have a separate ipf6.conf file. Both IPv4 and IPv6 packets can be used in the same file. For those that have separate files today, they should not interfere with each other unless you have "block in all" for IPv4 and "pass in all" for IPv6 or similar. In that case, the "block in all" will affect IPv6 traffic. This is a reflection of the internal design where there is now only a single list of filter rules, not one for each protocol. Check the man page for ipf.conf for more details.''

Hubertf's NetBSD Blog: EuroBSDcon 2011 NetBSD Presentation

Mon, 30 Jan 2012 20:57:00 +0000

spz@ gave a presentation with a status report of NetBSD during last EuroBSDCon. Slides in HTML format are available now - enjoy!

The Julipedia (Blog): Lutok 0.1 available

Julio Merino — Sun, 29 Jan 2012 21:48:05 +0000

A few months ago, I <a href="http://blog.julipedia.org/2011/09/introducing-lutok-lightweight-c-api-for.html">introduced the Lutok project</a>,&nbsp;a simple C++ API for Lua. To recap: the major goal of this API, which does not mimic the Lua C API bit by bit, is to enforce correct coding practices on the client side. This is done by (ab)using the RAII programming pattern to automatically free resources when not needed and to ensure that the Lua stack is correctly managed. The library also adheres to common C++ programming idioms and exposes exceptions for error management and uses the pimpl idiom to completely hide the Lua C API from clients of Lutok (unless you use the <tt>c_gate</tt> backdoor!).<br /><br />Today, I am pleased to announce that the first formal release of Lutok, obviously named 0.1, is available for download! You can obtain this release by visiting the <a href="http://code.google.com/p/lutok/downloads/detail?name=lutok-0.1.tar.gz">lutok-0.1.tar.gz download page</a>.<br /><br />Also, in preparation for this release, I have spent the weekend writing <a href="http://code.google.com/p/lutok/wiki/Examples">some little example programs</a> to demonstrate the usage of Lutok, and also some&nbsp;<a href="http://code.google.com/p/lutok/wiki/Installation">installation instructions</a>.<br /><br />Hope you find this release useful and please do send me any comments you may have!<br /><br />Hint: Yes, releasing Lutok 0.1 was a prerequisite for Kyua 0.3. So stay tuned ;-)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-283894938669140420?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: NetBSD vs. disk transfer speeds vs. BIOS settings

Sun, 29 Jan 2012 12:40:00 +0000

A few days ago, Brian Hoard made an interesting finding about performance of g4u, a NetBSD/i386-based disk cloning system. Citing from Brians mail:

``First, my problem was I had just replaced my motherboard on my custom build PC. Once I got Windows 7 64-bit loaded and everything working, I sat up to clone my system drive. The drive is a 500GB Seagate Barracude, SATA 2 drive. Cloning locally to an identical drive.

When booting into g4u, my transfer speeds were extremely slow. Normally, my 500 Gb clones take only about 90 minutes. But this was still working after over 6 hours. The g4u transfer speed was reporting only 1.5 Mb/sec.

I shut things down, and went into my system BIOS. I noticed that the SATA mode was set to "IDE Mode" for my drives. I changed this to "AHCI Mode" and continued to boot into g4u. This worked to fix the transfer speeds, and my clone finished normally. Getting 83 Mb/sec.

Once the drive was finished, I attempted to boot into Windows, but it would not boot. I had to change my BIOS back to "IDE Mode", then Windows behaved normally.

Upon researching this, I am now learning that you should enable AHCI Mode BEFORE installing Windows for it to work. Apparently, if Windows is not installed while using AHCI Mode, it disables the drivers for AHCI on the system drive. So if you later enable AHCI in your BIOS as I did, Windows will not have the driver loaded. I saw there is a fix on the Microsoft web site, but I haven't attempted to try it yet.

If someone else runs into a similar problem, hopefully this will help you.''

FWIW, g4u-2.5beta1 is based on NetBSD-current from January 2012, so checking your BIOS may help anyone seeing bad disk performance out there.
(Emphasizes in the text added by me)

Hubertf's NetBSD Blog: BSD Certification and the lack of training material - a call for participation! (Updated)

Sat, 28 Jan 2012 14:03:00 +0000

As you are aware, there's a BSD Associate Certification available from the BSD Certification group (that I'm a member of, working as subject matter expert for NetBSD). That's good!

There's also is a PDF which lists the BSD Associate (BSDA) examn objectives on 57 pages. That's neccessary!

There is currently no training material available that covers all the examn objectives, and that allows people interested in the certification to start learning. That sucks!

Now that's where I'd like to ask the NetBSD (and actually the whole BSD community) for support: This is not a small task, but I think it would be worthwhile for the whole community to have that available, either in closed (paper/book) or in public (electronic) form.

Any takers?

Update: Jeremy Reed reminds me that he has startet a Wiki-based approach to cover all topics of the BSDA, with the eventual goal to publish the result in book form. There is a Wiki-to-PDF transformation engine already in place, and the primary focus can be put on the contents at this point. So, this is the point where YOU come in. Have a look, get involved, participate in a novel project!

Hubertf's NetBSD Blog: BSDCan 2012 - call for papers (Updated)

Fri, 27 Jan 2012 15:57:00 +0000

Watching conference proceedings and publications like magazines, I feel a certain lack of NetBSD presence. Even in events that are BSD-friendly (EuroBSDcon, BSD Magazine come to mind). So here's a friendly reminder to go out on the street and preach the truth, as posted by Dan Langille on netbsd-advocacy@: You have two days left before the deadline!

Dan continues: ``BSDCan 2012 will be held 11-12 May, 2012 in Ottawa at the University of Ottawa. It will be preceded by two days of tutorials on 9-10 May.

NOTE: This will be Fri/Sat with tutorials on Wed/Thu.

We are now accepting proposals for talks.

The talks should be designed with a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.

If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal. Possible topics include:

How we manage a giant installation with respect to handling spam.
and/or sysadmin.
and/or networking.

From the BSDCan website, the Archives section will allow you to review the wide variety of past BSDCan presentations as further examples.
Both users and developers are encouraged to share their experiences.
The schedule is:
8 Jan 2012 Proposal acceptance begins
29 Jan 2012 Proposal acceptance ends
19 Feb 2012 Confirmation of accepted proposals
See also http://www.bsdcan.org/2012/papers.php
Instructions for submitting a proposal to BSDCan 2012 are available from: http://www.bsdcan.org/2012/submissions.php
Update: The deadline for submissions has been extended to Tuesday 31 January.

Hubertf's NetBSD Blog: Cobalt RestoreCD/RestoreUSB Beta based on NetBSD 5.1.1

Wed, 25 Jan 2012 23:10:00 +0000
Izumi Tsutsui writes on port-cobalt: ``It seems NetBSD 5.1.1 release is pending, but binaries are there and it also contains telnetd vulnerability fix (which is rather important for restorecd), so I'd announce 5.1.1 based NetBSD/cobalt RestoreCD and brandnew RestoreUSB as Beta test for future 5.1.x release:
http://ftp.NetBSD.org/pub/NetBSD/arch/cobalt/restore-cd/5.1.1/
restorecd-5.1.1-20120112.iso.gz is a gzipped RestoreCD ISO9660 image as prior releases.
restoreusb-5.1.1-20120112.img.gz is a new "RestoreUSB" image which has almost identical functions with RestoreCD but is intended to be burned into USB memory sticks for USB bootable PCs.
You can write the image using gzip(1) + dd(1) on Unix like OSes, or you can also use "Rawrite32" utility on MS Windows: http://www.NetBSD.org/~martin/rawrite32/index.html
To use the RestoreUSB for cobalt installation, write the image into >=512MB USB memory stick (or USB HDD etc.) and boot your PC from it, then all other procedures are same as RestoreCD. You no longer have to burn a coaster for every installation ;-)
See also "Restore CD Howto" for actual installation procedures: http://www.NetBSD.org/ports/cobalt/restorecd-howto.html: (though RestoreUSB is not mentioned yet) and see files in .tar.gz archive for more details. ''
Time to get out the good old Cobalt cube :)

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 23 Jan 2012 15:48:05 +0000
<br /><ul><li><a href="http://www.netbsd.org/~jmmv/atf/news.html#20120116-atf-0-15-released">Released ATF 0.15</a> and imported it into NetBSD.</li><li>Added support for integer/float printf-like modifiers to the <tt>utils::format</tt> module. These will be required to beautify size and time quantities in the reports and error messages.</li><ul><li>I spent way more time than I wanted on this. At first, I attempted to use <tt>std::snprintf</tt> to parse and process the format modifiers for integers and floats so that I could avoid implementing a custom parser for them. While this sounds like a cool idea (yay, code reuse!), it resulted in a ugly, nasty and horrible mess. In the end, I just ended up implementing custom parsing of the formatters, which was way easier and "good enough" for Kyua's needs.</li></ul><li>Started work on backporting ATF's new <tt>require.memory</tt> property into Kyua. This needs having a way to parse and format byte quantities in user-friendly forms (e.g. 1k, 2m, etc.)... hence the previous work on <tt>utils::format</tt>!</li><li>Set up a <a href="https://plus.google.com/b/109170076014992185386/">Google+ Page for Kyua</a>. I have no idea what to use it for yet. Maybe the status reports should go in there. Ideas?</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-2326868603928918712?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 16 Jan 2012 03:48:04 +0000
Finally some progress!<br /><br /><ul><li>Backported the <tt>require.memory</tt> changes in NetBSD to the ATF upstream code, and extended them to support OS X as well.</li><li>Backported local pkgsrc patches to ATF into the upstream code.</li><li>Started to prepare ATF 0.15 by doing test runs of NetBSD/i386 and NetBSD/amd64 and by building the code in various Linux distributions. Several build bugs fixed along the way.</li><li>Spent a long while trying to figure out how the Fedora package maintainer procedure has changed since 3 years ago to create packages for ATF, Lutok and Kyua. Not very successful yet unfortunately.</li></ul><div>Nothing on the Kyua front, but getting a new release of ATF out of the door has higher priority now!</div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-4383268139030266213?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: g4u 2.5beta1 supports handling of partition tables and bad disk sectors

Sat, 14 Jan 2012 13:48:00 +0000
After some absence (job-related) and technical problems (building of NetBSD failing for me from Mac OS X), I'm very happy to release a beta version of g4u with some long-overdue changes. Those include being able to backup/restore the MBR, which includes the partition table - needed when recovering single partitions to a new disk. Also, the various commands reading disks are now adjusted to not abort when a disk sector cannot be used. Instead, the bad bytes are skipped and the rest of the disk is recovered. Please give me feedback on this feature as I didn't have a bad disk to test this! Other news include a command to wipe a disk by completely overwriting it with 0-bytes (once). Last, command line editing was enabled - finally!
Remember that this is a test release, so your feedback is wanted - either to me in person, or to the g4u-help mailing list. Thanks!
Here's a full list of changes:

New commands "uploadmbr" and "slurpmbr" to backup and restore the master boot record, which includes the partition table. Required to restore a partition to an empty disk.
New command "copymbr" to copy the MBR from one disk to another, similar to "copydisk"
New command "wipedisk" to write the disk full with 0-bytes once from start (sector 0) to end (last sector)
Enable command line history/editing by forcing /bin/sh to be built without -DSMALL (ugly hack... there be lots of dragons!)
When setting up a fresh compile tree, g4u patches are now applied automagically without aborting the build
Error detection was now enabled in the dd(1) command, which is the core of g4u (surprise!). With that, disks with broken/unreadable sectors should now be copied, skipping the unreadable sectors and copying the rest. This affects a number of programs: copydisk, copypart, uploaddisk, uploadpart. BEWARE: I wasn't able to actually test this as I do not have a disk with bad sectors here. Please report back your experiences!!!
Make this build with NetBSD-current sources as of 2012-01-12
New drivers added to the kernel:

RDC PMX-1000 IDE controllers
Intel SCH IDE controllers
TOSHIBA PICCOLO controllers
Attansic/Atheros L1C/L2C Ethernet
Broadcom BCM43xx wireless
Agere/LSI ET1310/ET1301 Gigabit Ethernet
RDC R6040 10/100 Ethernet
USB LCDs and USB-VGA adaptors, e.g.:
DisplayLink DL-1x0/1x5
Option N.V. Wireless USB WAN modems
Microsoft RNDIS specifications USB ethernet
Atheros AR9001U USB Wifi
Intersil PrismGT USB Wifi
Virtio PCI, memory balloon, disk & network devices
... and many more that slipped past QA

... and any driver updates, optimizations and bug fixes and other enhancements from NetBSD-current

Get g4u 2.5beta1:

ISO
compressed ISO
First floppy
2nd floppy
3rd floppy
all 3 floppies
sources

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Wed, 11 Jan 2012 03:48:04 +0000
It's that time of the week again (although delayed by over a day). Unfortunately, no progress either during past week. Being on semi-vacations doesn't leave much "free time"... However, I traveled back home yesterday and getting back to my daily routine should give some spare time!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-3855575707225556807?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Tue, 03 Jan 2012 10:48:05 +0000
Happy new year!<br /><br />No activity past week as I was completely away from the computer.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-6092010260159605891?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: On the continuing decline of the GPL

Mon, 26 Dec 2011 21:21:00 +0000
Pointed out by Mishka, who found this 451 group's blog posting, I'll let the image speak on itself, even if it does not target NetBSD in particular:

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 26 Dec 2011 11:48:05 +0000
The post title should mention ATF instead of Kyua... but I'm keeping the usual one for consistency:<br /><br /><ul><li>Integrated timestamps into the XML and HTML reports generated by <tt>atf-report</tt>. These should soon show up in the continuous tests of NetBSD.</li><li>Work on integrating the use of POSIX timers into <tt>atf-run</tt> after Christos Zoulas performed these changes in the NetBSD tree. The result is quite awful because I need to keep compatibility with systems that do not provide the "new" interface...</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-9196312908393344096?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: The timing of security advisories

Sat, 24 Dec 2011 01:14:00 +0000
It's an old debate on when to release a security advisory: It should be released as early as possible to give people a chance to fix, but at the same time the fixing should be in a coordinated way. "Coordinated" means a fair chance for professional sysadmins to deploy a fix during working hours, and not in the middle of the night on a weekend. Or on the day before chistmas eve. But what if there's a pressing reason, maybe an exploit in the wild?
Apparently FreeBSD's telnetd currently has such a problem, and I think it's fair that Colin Percival as the FreeBSD Security Officer did release the advisory, even if it's in a sub-optimal timeframe.
For those NetBSD uses wondering if there's a similar problem in NetBSD's telnetd: Apparenly an unchecked argument can cause memory corruption by a memcpy length parameter overflow in sub-option processing (for terminal type, size etc.). This was fixed in NetBSD thanks to a hints from Colin. There's no NetBSD Security Advisory yet, but people still using telnetd in production networks may consider rebuilding libtelnet and telnetd.
So, to those of you who have moved to SSH: Happy Holidays!
To the rest: Happy Updating! :-)

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 19 Dec 2011 15:48:05 +0000
Unfortunately, not much activity this week due to travel reasons. Anyway, some work went in:<br /><br /><ul><li>Preliminary code to generate HTML reports from <tt>kyua report</tt>. This is easy peasy but boring. The current code was written as a proof of concept and is awful, hence why it was not committed. I'm now working in cleaning it up.</li><li>Backported test program and test case timestamping into ATF based on a patch from Paul Goyette. This is a very useful feature to have, and it will have to be added to Kyua later. (It has always been planned to be added, but have not had the time yet.)</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-113774404018064899?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: Two new NetBSD Security Advisories: OpenPAM and BIND resolver

Fri, 16 Dec 2011 13:01:00 +0000
The NetBSD Security Officers have released two new security advisories about problems found in 3rd party software that comes bundled with NetBSD's base system, OpenPAM and the BIND resolver.

NetBSD Security Advisory 2011-008: OpenPAM privilege escalation
Affects pkgsrc, all release branches and -current before 2011-11-09, it's fixed in all branches (current, netbsd-4 and -5) after that date.
Details from the advisory: ``The pam_start() function of OpenPAM doesn't check the "service" argument. With a relative path it can be tricked into reading a config file from an arbitrary location. NetBSD base utilities pass fixed constant strings. 3rd party programs which run with elevated privileges and allow user chosen strings open an attack vector.''
NetBSD Security Advisory 2011-009: BIND resolver DoS:
Affects pkgsrc, all release branches and -current from before 2011-11-20, it's fixed in the CVS branches after 2011-11-20.
Details from the advisory: ``Unpatched BIND 9 resolvers may cache an invalid record, subsequent queries for which could crash the resolvers with an assertion failure. ISC provided a patch which makes named recover gracefully from the inconsistency, preventing the abnormal exit.
The patch has two components. When a client query is handled, the code which processes the response to the client has to ask the cache for the records for the name that is being queried. The first component of the patch prevents the cache from returning the inconsistent data. The second component prevents named from crashing if it detects that it has been given an inconsistent answer of this nature.''

Happy updating!

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 12 Dec 2011 03:48:06 +0000
Some significant improvements this week:<br /><ul><li>Finally submitted the code to store and load full test case definitions. This is quite tricky (and currently very, very ugly) but it works and it will allow the reports to include all kinds of information from the test cases.</li><li>Removed the <tt>Atffile</tt>s from the tree; yay! For a long time, I had been using <tt>atf-run</tt> to run broken tests because <tt>atf-run</tt> allowed me to watch the output of the test case being debugged. However, this has been unnecessary since the <a href="http://blog.julipedia.org/2011/08/kyua-weekly-status-report_22.html">introduction of the <tt>debug</tt> command in late August</a>. I now feel confident that these files can go. (And <tt>debug</tt> is much more powerful than <tt>atf-run</tt> because you can target a single test case instead of a whole test program.)</li><li>Some crazy work attempting to hide the name of SQLite types from the <tt>sqlite::statement</tt> interface. I've been only able to do so somewhat decently for <tt>bind</tt>, but all my attempts at doing the same with <tt>column</tt> result in horrible code so far. So no, such changes have not been submitted.</li><li>As of a few minutes ago, <tt>kyua test</tt> now records the output of the test cases (stdout and stderr) into the database. These will be invaluable for debugging of test cases, particularly when the reports are posted online.</li><li>Some preliminary work at implementing HTML reports. This, however, has not received much progress due to the previous item requiring completion.</li></ul><div>I'm quite excited at this point. HTML reports are a few weeks away <i>at most</i>. Once that happens, it will be time to start considering replacing the <tt>atf-run</tt> / <tt>atf-report</tt> duo for good, particularly within NetBSD. This will certainly not be easy... but all the work that has gone into Kyua so far has this sole goal!</div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-7946776269603782071?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: BSDTalk Interview: Jim Brown on BSD Certification

Thu, 08 Dec 2011 14:39:00 +0000
Dru mentioned on Facebook that BSD Talk #209 is out, this time with Jim Brown, one of the key people behind the BSD certification.
BSD Certification covers NetBSD, FreeBSD, OpenBSD and DragonflyBSD, and has the goal to offer certification about BSD specific tasks in both Associate and Professional levels.
Based on my personal, most recent experience, certification is a good thing as you can show that you have a full grasp on a topic with all the relevant topics, and that you didn't only learn the few things that are relevant for your current job. Get certified!

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Tue, 06 Dec 2011 00:48:03 +0000
<br /><ul><li>Some more work towards allowing storing and loading of full test case definitions (which in turn will allow us to provide detailed HTML reports). I have local changes that do this, but they are gated by the lack of some additional tests and probably some optimizations, because they slow down <tt>kyua report</tt> significantly. Regarding commits, I have only submitted some related cleanup-changes.</li><li>I got distracted by invalid Doxygen comments and traced down why they were not being correctly validated (which is the whole point of running Doxygen during the build). It turns out I had to enable some extra <tt>EXTRACT_*</tt> settings in the <tt>Doxyfile</tt>. After doing so, I realized there are many, many documentation problems in the code... and I have been fixing them since. It's a tough operation, but I'm more than half-way through already. To give you an idea, the current diffstat shows about 650 new comment lines (!).</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-3760015699059977369?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 28 Nov 2011 15:48:05 +0000
My goal for this past week was to change the database layer to be able to store full definitions of the test cases, and to later be able to load these while scanning an action. This is to allow the <tt>report</tt> command to provide all kinds of information about the executed tests, not just their names and their results. However, adding this functionality has proven to be more complex than I wished because the current types to represent test programs and test cases are kinda broken: that the abstractions chosen a while ago do not seem to be appropriate, and this is complicating further changes.<br /><br />Due to this, I ended up doing some cleanups. First, I reimplemented the way in which test programs that fail to list their test cases are represented. And second, I got rid of the useless <tt>test_case_id</tt> class, which exposed even further problems in the data types that represent test cases.<br /><br />It's now time to sit and think if the current representations of test programs and test cases make sense and, if not, how to better redo them. Not going to be easy, but I hope to have some time for this cleanup during this upcoming week.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-8378862959071731158?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: Netbooting g4u via PXE

Thu, 24 Nov 2011 01:17:00 +0000
Doing a network based boot with PXE is not exactly hard, but you need some debugging and the right tools in place. If you want to netboot g4u, the NetBSD-based tool for harddisk image cloning via FTP, via PXE, there's a description on how to do Netbooting of g4u via PXE by Mariusz Zynel.
Details include setting up a TFTP server for loading the bootloader and getting DHCP sending out the right files.

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 21 Nov 2011 03:48:06 +0000
I only have one thing to report this week, but oh boy it's big: <b>the <tt>report</tt> command finally reports the results of a run of a test suite!</b> Yes, you heard well: Kyua is, finally, able to keep a record of all the previous executions of test suites and allows you to extract reports of any of them a posteriori.<br /><div><br /></div><div>At the moment, the report is just some disorganized plain-text. &nbsp;For example:</div><div><br /></div><div><pre>$ kyua test</pre><pre>[... wait for the tests to run ...]</pre><pre>Committed action 82</pre><pre>$&nbsp;kyua report<br />===&gt; Skipped tests<br />[... too long to show ...]<br />===&gt; Expected failures<br />integration/cmd_report_test:output__console__change_file &nbsp;-&gt;<br />&nbsp; &nbsp; expected_failure: --output not implemented yet:<br />&nbsp; &nbsp; atf-check failed; see the output of the test for details<br />===&gt; Failed tests<br />store/transaction_test:put_test_case__ok &nbsp;-&gt;<br />&nbsp; &nbsp; failed: Line 663: stmt.step() not met<br />===&gt; Summary<br />Action: 82<br />Test cases: 934 total, 15 skipped, 1 expected failures,</pre><pre>0 broken, 1 failed</pre></div><br />I'm now working on changing the database schema to be able to really store all the data about test cases, because at the moment I'm only storing their names. Once all the original data is stored, the <tt>report</tt> command will have lots more information to work with, and then will be the time to start improving the format of the reports. As mentioned earlier, having interactive HTML dashboards is high in the priority list, and a very important goal of Kyua altogether.<br /><br />Stay tuned :-)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-245230397741999971?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: Tanenbaum: Minix to become NetBSD-compatible

Fri, 18 Nov 2011 22:16:00 +0000
I've found this one via (german language) heise online: Andrew Tanenbaum, operating system researcher and father of the Minix operating system gave an interview to the french LinuxFr.org site. Topics include where Minix is today, and where it will move to in the future.
The latter one is worth quoting in this blog's NetBSD context: ``We are now focused on three things: NetBSD compatibility, embedded systems, and reliability. 3.2.0 will have a lot of headers, libraries, and userland programs take from NetBSD, which is a very stable, mature system. The BSDs are quite popular as you may know. One of them is sold under the brand name "Macintosh" by Apple. [...]
We think NetBSD is a mature stable system. Linux is not nearly as well written and is changing all the time. NetBSD has something like 8000 packages. That is enough for us.''
Further topics include multicore and microkernel architectures, grants from Google and the European Research Council, software licensing, the GPL and Linux. Have a look!

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 14 Nov 2011 16:48:06 +0000
Kyua has finally gained a <tt>report</tt> subcommand, aimed at processing the output data of an action (stored in the database) and generating a user-friendly report in a variety of formats. This is still extremely incomplete, so don't get your hopes too high yet ;-) The current version of the <tt>report</tt> command takes an action and all it does is dump its runtime context (run directory, environment variables, etc.). Consider it just a proof of concept.<div><br /></div><div>I have now started work on loading the data of test case results for a particular action, and once that is done, the <tt>report</tt> command will start yielding really useful data: i.e. it will actually tell you what happened during a particular execution of a test suite. The way I'm approaching the work these days is by building the skeleton code to implement the basic functionality first (which actually involves writing a lot of nasty code), with the goal of adding missing pieces later bit by bit.</div><div><br /></div><div>For example, at this moment I'm only targeting text-based outputs with a limited set of data. However, when that is done, adding extra data or different formats will be relatively easy. Generating HTML dashboards (without going through XML, as was the case of <tt>atf-report</tt>!) is definitely highly prioritized.</div><div><br /></div><div>By the way: I just realized it has already been one year since Kyua saw life. Wow, time flies. And only now we are approaching a point where killing the <tt>atf-run</tt> / <tt>atf-report</tt> pair is doable. I'm excited.</div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-2140157323321450898?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 07 Nov 2011 15:48:05 +0000
Many things have happened this week, but they can all be summarized in one single sentence: <tt><b>kyua test</b></tt><b> now records the results of the execution of a test suite into the SQLite database</b>.<br /><br />"Why is this important?", you ask. Well, that's a good question. Recording test results opens the gate to many long-awaited features that should be coming soon, such as the ability to inspect the history of a particular test, to query all available data of a test result and/or to generate a dashboard of test results. It's interesting to realize that <i>most of these features are just one SQL query away</i>. If you install Kyua, you can already run a few tests and then use <tt>kyua db-exec</tt> to issue arbitrary SQL queries against the database; the schema (see <tt>store/schema.sql</tt>) might look a bit convoluted, but a bunch of <tt>NATURAL JOIN</tt>s will yield the desired output.<br /><br />The feature requests that have the highest priority at this point are the ability to generate a report of the last tests run both as a text file and as an HTML dashboard, because having these features means we can finally kill the <tt>atf-run</tt> and <tt>atf-report</tt> pair. At this point I'm, once again, "stuck" while figuring out how to best organize the code to make all these things possible while still keeping a nice separation across the existing layers (<tt>cli</tt>, <tt>engine</tt> and <tt>store</tt>)... all without introducing much unnecessary complexity. But exciting times lie ahead!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5051367421950896129?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 31 Oct 2011 14:48:05 +0000
<br /><ul><li>Submitted a placeholder implementation of the new persistence layer (<tt>store</tt> top-level directory). This only supports opening a database and ensuring its metadata is valid. <a href="http://code.google.com/p/kyua/source/detail?r=253">See r253</a>.</li><li>Added a <tt>db-exec</tt> CLI command to execute arbitrary SQL commands onto the database. This is handy during development and testing, but may also help users to extract information out of the database in those cases where the CLI does not cover their needs just yet. <a href="http://code.google.com/p/kyua/source/detail?r=255">See r255</a>.</li><li>Miscellaneous fixes and improvements to <tt>utils::env</tt> and <tt>utils::sqlite</tt>.</li><li>Preliminary code to support putting objects (like actions and contexts) into the database. I've been thinking about this for a while and finally came up with a design that completely decouples the persistence needs from the higher-level classes in the <tt>engine</tt> layer. I haven't submitted the code yet though, as it lacks tests. (Still thinking how to write the loading of objects though.)</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5304169474623962697?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 24 Oct 2011 02:48:05 +0000
<br /><ul><li>Moved all the logic code of the "debug", "list" and "test" commands from the CLI layer to the engine layer.</li><ul><li>Up until now, the CLI modules implementing these commands contained all the logic to load Kyuafiles and iterating over them to find matching tests and applying the desired operation to them. This kind of code belongs in "driver" modules (aka controllers) of the engine layer, because there is nothing UI-related in them.</li><li>After this refactoring, the code left in the CLI modules is purely presentation-related, and the code in the engine implements all the logic.</li><li>The goal of these changes is to be able to hide the interactions with the database in these controllers. The CLI layer has no business in dealing with the database connection (other than allowing the user to specify which database to talk to, of course).</li></ul><li>Implemented a very simple RAII model for SQLite transactions.</li><li>Some additions to the utils::sqlite bindings to simplify some common calling patterns (e.g. binding statement parameters by name).</li><li>Preliminary prototypes at database initialization. This involves creating new databases and populating them with the initial schema, plus dealing with database metadata to, e.g. detect if we are dealing with the correct schema version.</li><ul><li>The code for this is still too crappy to be submitted, so don't look for it in the repository just yet!</li><li>The design document details many things that should be part of the schema (e.g. "sessions"0, but I've decided that I'll start easy with a simplified schema and later build on top of it. Otherwise there will be too many clunky moving parts to deal with while the fundamental ideas are not yet completely clear.</li></ul><li>Fixes to let the code build and run again in NetBSD (macppc at least).</li></ul><div>I've now been stuck for a few days trying to figure out what the best way to implement the conversion of (new) in-memory objects to database objects is, and how to later recover these objects. E.g. what the correct abstractions are to take test case results and put them in the database, and how to retrieve these results to generate reports later on. I now start to have a clear mental picture on how this should look like, but I have yet to see how it will scale.</div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-402139290656767976?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: Enlarging a (virtual) disk

Sat, 22 Oct 2011 21:04:00 +0000
I've tried to build NetBSD-current at various points in the past few months, and always hit one of two bugs: -current blows up with a gcc Internal Compiler Error when crossbuilding on Mac OS X, and kernel panics with native NetBSD builds with sources on NFS. This stinks, and I've successfully managed to do a successful -current build with sources on (local) disk. With NetBSD running within VMware Fusion on Mac OS X.
To go on from there, I found that my NetBSD VM's only disk was too small to do anything useful. Options for enlarging that came to mind:

NFS - see 'panic' above, no go.
Adding another (virtual) disk - easily doable, but I felt like not adding one
Extending the existing disk - adventure time!
Option #3 was it, and after removing all VMware snapshots, enlarging the disk was easy with VMware Fusion, going from 10GB to 20GB. After growing the disk itself, the next question was how to use the newly gained disk. Of course some file system needs to use it, and in theory there are the following options:

Enlarge the last file system on disk
Fix the partition table to add another partition for the new space
The disk was resized from 10GB to 20GB. The partition table (disklabel) was created by a standard NetBSD install, and first had the root file system, followed by the swap partition. From that, adding 10GB more swap was not useful, so I've decided to change the disklabel to add the new disk space as a new partition behind the existing partitions. This is also an excuse to not frob with growfs and resize_ffs. (And of course I'm ignoring the option of backing up the full file system, doing a full rebuild of the filesystem and then doing a restore :-)
For those in a similar situation, here are the steps to use the newly gained space on an enlarged (virtual) disk:

Prepare: save the old output of "dmesg" (/var/run/dmesg.boot is OK)

Enlarge - VMware Fusion wants a shutdown for that, you cannot suspend the machine

After booting, run a diff on the saved "dmesg" output, to learn what the old and new size of the disk is, in sectors. My diff looks like this, note the size change in sectors:
-wd0: 10240 MB, 22192 cyl, 15 head, 63 sec, 512 bytes/sect x 20971520 sectors +wd0: 20480 MB, 44384 cyl, 15 head, 63 sec, 512 bytes/sect x 41943040 sectors

Backup the existing/old disklabel, just in case: disklabel wd0 >disklabel.BAK

Edit the disklabel: disklabel -e wd0

In the editor, adjust the disk size in sectors from 20971520 to 41943040:
total sectors: 41943040

Partition 'd' is the full disk on i386/amd64, it starts at sector 0 and is 41943040 sectors big
# size offset fstype [fsize bsize cpg/sgs] d: 41943040 0 unused 0 0 # (Cyl. 0 - 44384*)

Partition 'c' is the NetBSD part of the disk. As this VM only has NetBSD, all the usable space is used. Note that "usable" space excludes the first 63 sectors of the disk (mbr etc.), i.e. it is 41943040 - 63 = 41942977 sectors:
# size offset fstype [fsize bsize cpg/sgs] c: 41942977 63 unused 0 0 # (Cyl. 0*- 44384*)

After this everything is in sync with the new disk again, and the remaining/new space can be used for new partition 'e'. As the new space starts where the disk used to end, its offset is the old size, 20971520 sectors.
The size of the new partition expands from the offset sector 20971520 to the end of the disk at sector 41943040, i.e. the partition size is:
% expr 41943040 - 20971520 20971520
In total, this gives for the new partition:
# size offset fstype [fsize bsize cpg/sgs] e: 20971520 20971520 4.2BSD 2048 16384 0 # (Cyl. 22192*- 44384*)

Last, create file system, mount and populate:
# newfs /dev/rwd0e # mkdir /disk2 # echo '/dev/wd0e /disk2 ffs rw,log 2 2' >>/etc/fstab # mount /disk2 # cd /usr ; pax -rw -pe -v stuff /disk2 # rm -fr stuff ; ln -s /disk2/stuff .

Now let's see if I get things far enough to get a build of g4u going... wish me luck!
P.S.: I'm offering choccolate to anyone fixing crossbuilding of NetBSD-current from Mac OS X. Any takers?

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 17 Oct 2011 03:48:06 +0000
<br /><ul><li>Added support to prepare and execute statements to <tt>utils::sqlite</tt>.</li><li>Added the <tt>UTILS_PURE</tt> definition to tag functions as pure. I should now sweep through old code to apply the attribute where possible.</li><li>Created a pkgsrc package for Vera++. Investigating if I can use this tool for coding style validation, as the current code of Kyua is a bit messy in this regard.</li><li>Made a quick attempt at getting <tt>kyua test</tt> record test results in a simple database; success! This was just a proof of concept, so it is not submitted yet.</li><li>Started to refactor the code to move many logic constructions from the <tt>cli</tt> to the <tt>engine</tt>. With the need to store test results in the database, it's clear that these semantics do not belong in the CLI, but the current code structure do not easily allow this. Need to add some "controller" classes in the <tt>engine</tt> to hide all the interaction among internal components.</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-4275543087243170349?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Sun, 09 Oct 2011 23:48:04 +0000
<br /><ul><li>Some more reading on SQLite. The <a href="http://www.amazon.com/Using-SQLite-Jay-Kreibich/dp/0596521189">Using SQLite</a> book is close to awesome; very easy to read and full of insightful tips.</li><li>Cleaned up my preliminary SQLite wrapper code. Still very incomplete, but I've bitten the bullet and decided to commit the new library as is; otherwise I'll just keep&nbsp;procrastinating. So, ladies and gentlemen, welcome <a href="http://code.google.com/p/kyua/source/detail?r=228">the new utils::sqlite module in Kyua</a>. At the moment, this just provides the barebones to open and close a database. I need to start playing around with preparing statements before I can model these in my wrapper classes. (Yes, the API structure is extremely similar to that in <a href="http://code.google.com/p/lutok/">Lutok</a>.)</li></ul><div><b>Update (23:40)</b>: I sent this too early, thinking I would not have anything nice to report this week either. But as it turns out, I finally had some spare time this late evening and got a chance to submit the extremely-incomplete SQLite C++ bindings I've been toying around with. See the update above! :-)</div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5370918960000243408?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 03 Oct 2011 19:48:06 +0000
<ul><li>Unfortunately, not much time this week either :-(</li><li>I am currently working on some adjustments to the design document of the database to describe new ideas; the previous design was incomplete in some areas and/or not accurate enough to support the described use cases.&nbsp; However, I've not had the to time to finish these edits and publish them.</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-2063946456167536880?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report

Julio Merino — Mon, 26 Sep 2011 22:48:05 +0000
Unfortunately, no activity this week other than some brainstorming on the database design.<br /><br />Why? My shipment container from Dublin arrived and I spent most of the weekend organizing stuff, setting up my little NetBSD box and attending a friend's wedding!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1017676619421586594?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): VirtualBox and port forwarding (and Fusion)

Julio Merino — Mon, 19 Sep 2011 22:48:05 +0000
I have been a <a href="http://www.vmware.com/products/fusion/overview.html">VMware Fusion</a> user for a long time and I think I am about to be a <a href="http://www.virtualbox.org/">VirtualBox</a> convert. Let me explain you my story.<br /><br />With the release of the <a href="http://msdn.microsoft.com/en-us/windows/home/br229518">Windows 8 Developer Preview</a>, I felt like giving the system a try. Not because I particularly like or care about Windows, but just because I am genuinely curious about operating systems and wanted to check out their new Metro interface. So I went ahead, downloaded the ISO image, created a new virtual machine under Fusion 3, started the virtual machine, and saw Fusion crash while the Windows installer was booting.<br /><br />Oh well, an emulation problem. I tried to fiddle with the virtual machine settings a bit, selecting different CPU virtualization settings and different hardware devices. No luck. After that, I resorted to Google to look for a solution to the problem and found that other people were experiencing the same issue. But, at the same time, I also found that VMware had just released Fusion 4 and that this new release works with Windows 8 just fine. Ah, cool! Or not... $50 for the upgrade. For my use cases (which amount to booting a few BSD and Linux virtual machines in console mode), it is hard to justify the upgrade — specially when it would just be to fulfill my curiosity and then delete the virtual machine 10 minutes later.<br /><br />But wait! There is that thing called VirtualBox. I had used this product a while ago and did not enjoy doing so because its interface was utterly (really) confusing. It was nowhere near the polish of Fusion or Parallels. However, to my surprise, most of my major itches have been fixed and the new interface of VirtualBox is bearable. And it boots the Windows 8 preview just fine!<br /><br />Right. So after installing Windows 8, playing with it for 10 minutes, and later deleting it, I decided to look around and see what else VirtualBox has to offer. It has <i>many</i> settings. Yes, I know, Fusion has many settings too, but most are hidden from the UI and can only be accessed by modifying the <tt>vmx</tt> text file. In the case of VirtualBox they are easily accessible.<br /><br />I kept looking around... and there it was, the "<b>Port Forwarding</b>" option staring at me under the Network settings.<br /><br />One of the features I've been wishing for a long, long, <i>long</i> time is the ability to easily configure a NATed (not bridged) virtual machine in my laptop, assign a host name to it from the graphical interface and then be able to SSH into the virtual machine from the laptop. As it turns out, this is doable in Fusion but it requires some nasty tinkering: one has to edit a <i>dhcpd</i> configuration file by hand to add a new stanza for the virtual machine (<tt>/Library/Application Support/VMware Fusion/vmnet8/dhcpd.conf</tt>). This involves manually copy/pasting the MAC address of the virtual machine into that file (which you must first extract from the <tt>vmx</tt> file), assigning it an IP in the corresponding subnet, and sticking the IP plus host name in the <tt>/etc/hosts</tt> file. This gets boring pretty quickly.<br /><br />VirtualBox may or may not support this kind of setting, but it has the aforementioned "Port Forwarding" option which is interesting on its own. As you can imagine, this sets up forwarding of a port from the host system to a port of the virtual machine. Creating an entry for SSH is as easy as clicking a couple of buttons, choosing a local port on the host system (e.g. 2022) and forwarding it to port 22 of the guest system. Voila. You can now access your virtual machine locally <i>and remotely</i> without having to go through any of the host name nor DHCP madness I experienced before:<br /><br /><tt>$ ssh -p 2022 localhost</tt><br /><br />You can later create an alias in your <tt>~/.ssh/config</tt> file to make accessing the virtual machine a breeze:<br /><br /><tt>Host myvm</tt><br /><tt>Hostname localhost</tt><br /><tt>Port 2022</tt><br /><br />And later:<br /><br /><tt>$ ssh myvm</tt><br /><br />It couldn't be easier. This trivial feature is a killer for the kind of environment I am interested in. And I also know that VirtualBox provides some other interesting features, like headless mode, that are not (easily) available with Fusion.<br /><br />So yes, I think I am a convert.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-6942172427809398844?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Kyua: Weekly status report (db designdoc edition!)

Julio Merino — Sun, 18 Sep 2011 23:48:03 +0000
<ul><li>Moved the code of <tt>utils::lua</tt> to a new project, <a href="http://code.google.com/p/lutok/">Lutok</a>.</li><li>Attempted to integrate a copy of Lutok into the Kyua source code to simplify installing Kyua. I have been playing with Subversion externals and Autoconf/Automake hacks to make this happen, but unfortunately haven't got a pleasant solution yet.</li><li>Modified Lutok to not expose the Lua C API at all from header files and cleaned up the Kyua code to cope with the changes.</li><li>Been chewing through the first chapters of the "<a href="http://www.amazon.com/Using-SQLite-Jay-Kreibich/dp/0596521189">Using SQLite</a>" book to refresh my SQL skills. </li><li>And, most importantly, wrote a <a href="http://code.google.com/p/kyua/wiki/DatabaseDesign">preliminary design document for the database store</a> of Kyua and the reporting features. Comments certainly welcome! Be aware that this is how <tt>atf-report</tt> will be replaced, so once this is done we should be able to finally kill <tt>atf-run</tt> and <tt>atf-report</tt> altogether :-)</li></ul><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-7580227422862832024?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Name your C++ auto-cleaners

Julio Merino — Sun, 18 Sep 2011 03:48:05 +0000
As you may already know, RAII is a very powerful and popular pattern in the C++ language. With RAII, you can wrap non-stack-managed resources into a stack-managed object such that, when the stack-managed object goes out of scope, it releases the corresponding non-stack-managed object. <a href="http://onlamp.com/pub/a/onlamp/2006/05/04/smart-pointers.html?page=1">Smart pointers</a> are just one example of this technique, but so are IO streams too.<br /><br />Before getting into the point of the article, bear with me for a second while I explain what the&nbsp; <a href="http://code.google.com/p/lutok/source/browse/trunk/stack_cleaner.hpp"><tt>stack_cleaner</tt></a> object of <a href="http://code.google.com/p/lutok/">Lutok</a> is. The "stack cleaner" takes a reference to a Lua state and records the height of the Lua stack on creation. When the object is destroyed (which happens when the declaring function exits), the stack is returned to its previous height thus ensuring it is clean. It is always a good idea for a function to prevent side-effects by leaving its outside world as it was — and, like it or not, the Lua state is part of the outside world because it is an input/output parameter to many functions.<br /><br />Let's consider a piece of code <i>without</i> using the stack cleaner:<br /><br /><pre>void<br />my_function(lutok::state&amp; state, const int foo)<br />{<br />&nbsp;&nbsp;&nbsp; state.push_integer(foo);</pre><pre>&nbsp;&nbsp;&nbsp; ... do something else in the state ...<br />&nbsp;&nbsp;&nbsp; const int bar = state.to_integer();<br />&nbsp;&nbsp;&nbsp; if (bar != 3) {<br />&nbsp; &nbsp; &nbsp; &nbsp; state.pop(1); <br />&nbsp; &nbsp; &nbsp; &nbsp; throw std::runtime_error("Invalid data!");<br />&nbsp; &nbsp; }<br />&nbsp; &nbsp; state.pop(1);<br /><br />}</pre><br />Note that we have had to call <tt>state.pop(1)</tt> from "all" exit points of the function to ensure that the stack is left unmodified upon return of <tt>my_function</tt>. Also note that "all exit points" may not be accurate: in a language that supports exceptions, any statement may potentially raise an exception so to be really safe we should do:<br /><br /><pre>void<br />my_function(lutok::state&amp; state, const int foo)<br />{<br />&nbsp;&nbsp;&nbsp; state.push_integer(foo);<br />&nbsp; &nbsp; try { <br />&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; ... do something else in the state ...<br />&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; const int bar = state.to_integer();<br />&nbsp;&nbsp; &nbsp; &nbsp;&nbsp; if (bar != 3 <br />&nbsp; &nbsp;&nbsp; &nbsp; &nbsp; &nbsp; throw std::runtime_error("Invalid data!");<br />&nbsp; &nbsp; } catch (...) {<br />&nbsp; &nbsp; &nbsp; &nbsp; state.pop(1);<br />&nbsp; &nbsp; &nbsp; &nbsp; throw;<br />&nbsp;&nbsp;&nbsp; }<br />&nbsp;&nbsp;&nbsp; state.pop(1);<br />}</pre><br />... which gets old very quickly. Writing this kind of code is error-prone and boring.<br /><br />With an "auto-cleaner" object such as the <tt>stack_cleaner</tt>, we can simplify our code like this:<br /><br /><pre>void<br />my_function(lutok::state&amp; state, const int foo)<br />{<br />&nbsp; &nbsp; lutok::stack_cleaner cleaner(state);<br /><br />&nbsp;&nbsp;&nbsp; state.push_integer(foo);<br />&nbsp;&nbsp;&nbsp; ... do something else in the state ...<br />&nbsp;&nbsp;&nbsp; const int bar = state.to_integer();<br />&nbsp;&nbsp;&nbsp; if (bar != 3) <br />&nbsp; &nbsp; &nbsp; &nbsp; throw std::runtime_error("Invalid data!"); <br />}</pre><br />And we leave the boring task of determining when to actually call <tt>state.pop(1)</tt> to the compiler and the runtime environment. In this particular case, no matter how the <tt>my_function</tt> terminates, we ensure that the Lua stack will be left as the same size as it was before.<br /><br />But, as I said earlier, all this was just an introduction to the idea that made me write this post.<br /><br />When you declare an auto-cleaner object of any kind, <i>be sure to give it a name</i>. It has happened to me a few times already that I have written the following construct:<br /><br /><pre>lutok::stack_cleaner(state);</pre><br />... which is syntactically correct, harmless and "looks good" if you don't look closely. The compiler will chew along just fine because, even though we are declaring an anonymous object, its constructor and destructor may be doing who-knows-what, so their code must be called and thus the "unused variable" warning cannot really be raised.<br /><br /><b>However</b> this does not give us the desired behavior. The cleaner object will be constructed and destructed in the same statement without having a chance to wrap any of the following code, because its scope is just the statement in which it was defined. In other words, the cleaner will have absolutely no effect on the rest of the function and thus will be useless.<br /><br />So, moral of the story: always give a name to your auto-cleaner objects so that their scope is correctly defined and their destructor is run when you actually expect:<br /><br /><pre>lutok::stack_cleaner ANY_NAME_HERE(state);</pre><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-7544613249446930628?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Introducing Lutok: A lightweight C++ API for Lua

Julio Merino — Thu, 15 Sep 2011 19:48:06 +0000
It has finally happened. <a href="http://code.google.com/p/lutok/">Lutok</a> is the result of what was promised in the "<a href="http://blog.julipedia.org/2011/09/splitting-utilslua-from-kyua.html">Splitting utils::lua from Kyua</a>" web post.<br /><br />Quoting the project web page:<br /><blockquote>Lutok provides thin C++ wrappers around the Lua C API to ease the interaction between C++ and Lua. These wrappers make intensive use of RAII to prevent resource leakage, expose C++-friendly data types, report errors by means of exceptions and ensure that the Lua stack is always left untouched in the face of errors. The library also provides a small subset of miscellaneous utility functions built on top of the wrappers.<br /><br />Lutok focuses on providing a clean and safe C++ interface; the drawback is that it is not suitable for performance-critical environments. In order to implement error-safe C++ wrappers on top of a Lua C binary library, Lutok adds several layers or abstraction and error checking that go against the original spirit of the Lua C API and thus degrade performance.<br /><br />Lutok was originally developed within Kyua but was later split into its own project to make it available to general developers.</blockquote>Coming up with a name for this project was quite an odyssey, and is what has delayed is release more than I wanted. My original candidate was "luawrap" which, although not very original, was to-the-point and easy to understand. Unfortunately, that name did not clear with the legal department and I had to propose several other names, some of which were not acceptable either. Eventually, I settled with "Lutok", which comes from "LUa TOolKit".<br /><br />At this point, the source tree of Lutok provides pretty much the same code as the <tt>utils::lua</tt> module of Kyua. While it may be enough to get you started, I'm pretty sure you will lack some functions in the <tt>state</tt> class. If that is the case, don't hesitate to file a bug report to let me know what is missing.<br /><br />In case you missed the link above, the project page is here: <a href="http://code.google.com/p/lutok/">Lutok in Google Code</a>. <div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-7627076009086318140?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Using va_copy to safely pass ap arguments around

Julio Merino — Tue, 13 Sep 2011 02:48:05 +0000
A long time ago, while I was preparing an ATF release, I faced many failing tests and crashes in one of the platforms under test. My memory told me this was a problem in OpenSolaris, but the <a href="http://www.blogger.com/mtn-host.prjek.net/viewmtn/atf/revision/info/655c6c51f2155076f482a038042c67fd25adc934">repository logs</a> say that the problem really happened in Fedora 8 x86_64.<br />The problem manifested itself as segmentation faults pretty much everywhere, and I could trace such crashes down to pieces of code like the following, of which the C code of ATF is full of:<br /><pre>void<br />foo_fmt(const char *fmt, ...)<br />{<br /> va_list ap;<br /><br /> va_start(ap, fmt);<br /> foo_ap(fmt, ap);<br /> va_end(ap);<br />}<br /><br />void<br />foo_ap(const char *fmt, va_list ap)<br />{<br /> char buf[128];<br /><br /> vsnprintf(buf, sizeof(buf), fmt, ap);<br /><br /> ... now, do something with buf ...<br />}</pre>The codebase of ATF provides <tt>_fmt</tt> and <tt>_ap</tt> variants for many functions to give more flexibility to the caller and, as shown above, the <tt>_fmt</tt> variant just relies on the <tt>_ap</tt> variant to do the real work.<br />Now, the crashes that appeared from the code above seemed to come from the call that consumes the <tt>ap</tt> argument, which in this case is <tt>vsnprintf</tt>. Interestingly, though, all the tests in other platforms but Linux x86_64 worked just fine, and this included OpenSolaris, other Linux distributions, some BSDs and even different hardware platforms.<br />As it turned out, you <b>cannot blindly pass <tt>ap</tt> arguments around</b> because they are not "normal" parameters (even though, unfortunately, they look like so!). In most platforms, the <tt>ap</tt> element will be just an "absolute" pointer to the stack, so passing the variable to an inner function calls is fine because the caller's stack has not been destroyed yet and, therefore, the pointer is still valid. But... the <tt>ap</tt> argument can have other representations. It'd be an offset to the stack instead of a pointer, or it'd be a data structure that holds all the variable parameters. If, for example, the <tt>ap</tt> argument held an offset, passing it to an inner function call would make such offset point to "garbage" because the stack would have been grown due to the new call frame. (I haven't investigated what specific representation is x86_64 using.)<br />The solution is to use the <tt>va_copy</tt> function to generate a new <tt>ap</tt> object that is valid for the current stack frame. This is easy, so as an example, we have to rewrite the <tt>foo_ap</tt> function above as follows:<br /><pre>void<br />foo_ap(const char *fmt, va_list ap)<br />{<br /> char buf[128];<br /> va_list ap2;<br /><br /> va_copy(ap2, ap);<br /> vsnprintf(buf, sizeof(buf), fmt, ap2);<br /> va_end(ap2);<br /><br /> ... now, do something with buf ...<br />}</pre>This duplication of the <tt>ap</tt> argument pointing to the variable list of arguments ensures that <tt>ap2</tt> can be safely used from the new stack frame.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-2547470671733170941?l=blog.julipedia.org' alt='' /></div></content>