BSD Planet * BSD People

FreeBSD Diary: 3Ware Nagios plugin

Thu, 02 Sep 2010 23:48:07 +0000

I liked it, but I wanted more

Hubertf's NetBSD Blog: BSD Professional (BSDP) Certification Exam Objectives published

Wed, 01 Sep 2010 21:28:00 +0000

In its mission to create certifications for BSD Un*x systems, the BSD Certification Group has now published the "BSD Professional" (BSDP) certification exam objectives:

`` The 78 page document outlines 11 knowledge domains, each containing a number of objectives, that form the basis of the certification. The document, a collaborative effort by members of the BSD community, is a followup effort to the BSDP Job Task Analysis report published earlier this year.

"The BSDP Certification Requirements Document provides a comprehensive set of objectives that a BSD Professional should master, ranging from general system administration to advanced security and system maintenance," said Babak Farrokhi, a seasoned BSD administrator and author of 'Network Administration with FreeBSD 7'.

Set to launch in 2011, the BSD Professional certification exam has been in development for over two years. Since the launch of the BSD Associate exam in 2008, there has been a continual effort to put together a certification for the next level of achievement for BSD system administrators.

"The BSD Professional certification is aimed at assessing practical skills of senior level BSD system administrators," said Jim Brown, BSDCG board member. "With the publication of these objectives, the BSD Certification Group is aiming to produce a high quality certification, giving the computing industry a new standard for excellence in system administration."

The BSDP Requirements document is available as a downloadable PDF in English''

The publication of the BSDP exam objectives is the next step after the BSD Associate (BSDA) certification exam objectives were published in 2005, which was then followed by quite a number of successful BSDA certifications. See the BSD Certification group's website for more details!

FreeBSD Diary: Where is my SCSI card?

Sun, 29 Aug 2010 23:48:05 +0000

Things are hidden right before your eyes

FreeBSD Diary: cssh - cluster ssh

Fri, 20 Aug 2010 00:48:07 +0000

Sometimes you need more than one

Hubertf's NetBSD Blog: New NetBSD Core Team announced

Mon, 16 Aug 2010 22:30:00 +0000

NetBSD's core team is responsible for technical steering of the project. In the past the group was five people, and it was increased to seven people now. Read more in Alistair Crooks' announcement:

``We take great pleasure in announcing that the NetBSD core team, responsible for technical management within the NetBSD project, has increased its numbers to seven. This is to help in the running of a project with an ever-growing source base and developer community, and mirrors a similar change made to the board of directors, which has worked extremely well.

To help with the running of the project, we have asked Antti Kantee (pooka%NetBSD.org@localhost) and Chuck Silvers (chs%NetBSD.org@localhost), and they have very kindly agreed to join the core team. Antti is well known to many both inside and outside the project, and has contributed many new and exciting ideas, the most memorable and useful of these being the rump kernel architecture. Chuck is also well-known in NetBSD circles - his work on UBC, and his recent update of the Linux emulation code are just two examples of his contributions.

We therefore thank them both for their outstanding work to date, and to their joining the core team to lead and guide progress in the future.

For the current core team:

Alistair Crooks -- agc%NetBSD.org@localhost
Matt Green -- mrg%NetBSD.org@localhost
Antti Kantee -- pooka%NetBSD.org@localhost
Chuck Silvers -- chs%NetBSD.org@localhost
Yamamoto Takashi -- yamt%NetBSD.org@localhost
Matt Thomas -- matt%NetBSD.org@localhost
Christos Zoulas -- christos%NetBSD.org@localhost

Hubertf's NetBSD Blog: Of course it runs ... 2.11BSD (or: PDP-11 in a FPGA)

Mon, 09 Aug 2010 22:55:00 +0000

It's about time NetBSD gets the PDP11 port done: the PDP-11/70 CPU core is now available as implementation on a FPGA-board, and there's need for a newer operating system than 2.11BSD! Citing from the homepage:

``The project contains a complete PDP-11 system: a 11/70 CPU with memory management unit, but without floating point unit, a basic set of UNIBUS peripherals (DL11, LP11, PC11, RK11/RK05), and last but not least a cache and memory controllers for SRAM and PSRAM. The design is FPGA proven, runs currently on Digilent S3BOARD and NEXYS2 boards and boots 5th Edition UNIX and 2.11BSD UNIX. ''

Hubertf's NetBSD Blog: Janitorial code maintenance with coccinelle (Updated)

Sun, 08 Aug 2010 23:22:00 +0000

According to its homepage, coccinelle is ``a program matching and transformation engine which provides the language SmPL (Semantic Patch Language) for specifying desired matches and transformations in C code. Coccinelle was initially targeted towards performing collateral evolutions in Linux. Such evolutions comprise the changes that are needed in client code in response to evolutions in library APIs, and may include modifications such as renaming a function, adding a function argument whose value is somehow context-dependent, and reorganizing a data structure.''

Jean-Yves Migeon has applied coccinelle to NetBSD, and suggested performing two manual tasks, citing from his email to tech-kern:

replace all sizeof(fooarray)/sizeof(foo) calculations by its __arraycount() macro equivalent, provided in cdefs.h
detect places where aprint_* calls could be replaced by aprint_*_dev() to save manual printing of the device driver's name

Implementation of the coccinelle scripts, resulting patched for NetBSD and more information can be found in Jean-Yves Migeon's posting, his list of patches, and on his wiki page.

Update: Jean-Yves' follow-up includes further examples:

logical not with bitwise "&"

NULL-pointer dereference after first checking if it's NULL(!?), my favourite (example diff):

 	if (xfer == NULL) {
-		aprint_error_dev(xfer->fc->bdev, "xfer == NULL\n");
+		aprint_error("%s: xfer == NULL\n", __func__);
 		return;
 	}

FreeBSD Diary: ZFS: Resizing your zpool

Tue, 03 Aug 2010 23:48:06 +0000

Got space? Need more?

FreeBSD Diary: ZFS: do not give it all your HDD

Sun, 01 Aug 2010 21:48:06 +0000

Sometimes, holding out is the best decision.

Hubertf's NetBSD Blog: NYCBSDCon 2010 Call for Papers

Wed, 14 Jul 2010 21:13:00 +0000

Citing from the call for papers: ``The New York City BSD Conference (NYCBSDCon) is the main technical conference on the US East Coast for the BSD community to get together to share and gain knowledge, to network with like-minded people, and to have fun. This event is organized by members of the New York City *BSD Users Group (NYC*BUG).
The NYCBSDCon program committee is accepting submissions for imaginative, embryonic and energizing presentations surrounding the BSD operating systems. We are looking to attract a wide range of speakers and attendees; therefore, topics of interest range from the esoteric to development to practical, everyday sysadmin life. Of course, original topics are preferred in most cases.
Each talk is expected to be 45-50 minutes, including a few minutes for questions and answers. All presentations will be recorded for audio and video. Presenters will have audio/visual and network connectivity.
Abstracts for presentations are due July 31, 2010.
Authors of accepted submissions should be able to provide the full presentation for publication on NYCBSDCon sponsored mediums. Further instructions will follow notification of acceptance. Submissions accompanied by a non-disclosure agreement or a product advertisement will be rejected.
Abstract submissions should be emailed to cfp@nycbsdcon.org in text, ps or pdf format.

Conference Location: Cooper Union, New York, NY Conference Dates: November 12-14, 2010''

Submission of NetBSD related entries is highly appreciated! See the call for papers for more information on important milestones, subsidizing of speakers and the mailing list to stay upto-date.

Dru Lavigne @ O'Reilly Network: Dru Lavigne at Summercamp - Aug 14 2010

Dru Lavigne — Tue, 06 Jul 2010 20:46:00 +0000

Dru Lavigne (BSD Hacks) will be presenting "Getting Started in an Open Source Community."

The Julipedia (Blog): ATF 0.10 released

Julio Merino — Fri, 02 Jul 2010 15:48:05 +0000

Ladies and gentlemen: I have just released ATF 0.10! This release with such a magic number includes lots of new exciting features and provides a much simplified source tree.<br /><br />Dive into the <a href="http://www.netbsd.org/~jmmv/atf/releases/0.10/">0.10 release page</a> for details!<br /><br />I'm now working in getting this release into the NetBSD tree to remove some of the custom patches that have been superseded by the official release. Will be there soon.<br /><br />And all this while I am a <a href="http://www.meetbsd.org/">meetBSD</a> in Kraków :-)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-4316464728404819592?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: BSD Magazine archive available

Thu, 01 Jul 2010 00:02:00 +0000

Olga Kartseva writes: ``BSD Magazine archives available without subscribing to BSDMag newsletter for freebsd-announce subscribers!'' Here are direct PDF links:

Enjoy - and remember: more NetBSD content is good content, authors are always welcome!

The Julipedia (Blog): Testing NetBSD: Easy Does It

Julio Merino — Thu, 24 Jun 2010 08:48:06 +0000

Antti Kantee has been, for a while, writing unit/integration tests for the puffs and rump systems (for which he is the author) shipped with NetBSD. Recently, he has been working on fixing the NetBSD test suite to report 0 failures in the i386 platform so as to encourage developers to keep it that way while doing changes to the tree. The goal is to require developers to run the tests themselves before submitting code.<br /><br />Antti has just published an introductory article, titled <a href="http://blog.netbsd.org/tnf/entry/testing_netbsd_easy_does_it">Testing NetBSD: Easy Does It</a>, that describes what <a href="http://www.NetBSD.org/~jmmv/atf/">ATF</a> and <a href="http://www.gson.org/netbsd/anita/">Anita</a> are, how to use them and how they can help in NetBSD development and deployment. Nice work!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1741546407166795638?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: NetBSD 5.1_RC3 binaries available for download

Sun, 20 Jun 2010 22:36:00 +0000

NetBSD release-engineer Soren Jacobsen announces: ``The third (and hopefully final) release candidate of NetBSD 5.1 is now available for download at:
http://ftp.NetBSD.org/pub/NetBSD/NetBSD-5.1_RC3/
Those of you who prefer to build from source can continue to follow the netbsd-5 branch, but the netbsd-5-1-RC3 tag is available as well.
See src/doc/CHANGES-5.1 for the list of changes from RC2 to RC3.

Please help us test this release candidate as much as possible. Remember, any feedback is good feedback. We'd love to hear from you, whether you've got a complaint or a compliment. That said, we hope your feedback is positive, as we would like this to be the final release candidate before 5.1. ''

The Julipedia (Blog): ATF 0.9 released (late announcement)

Julio Merino — Fri, 18 Jun 2010 18:48:05 +0000

Oops! Looks like I forgot to announce the <a href="http://www.netbsd.org/~jmmv/atf/news.html#20100603-atf-0-9-released">release of ATF 0.9</a> here a couple of weeks ago. Just a short notice that the formal release has been available since June 3rd and that 0.9 has been in NetBSD since June 4th!<br /><br />You can also enjoy a <a href="http://www.netbsd.org/~jmmv/atf/">shiny-new web site</a>! It even includes a <a href="http://www.netbsd.org/~jmmv/atf/docs/faq.html">FAQ</a>!<br /><br />And, as a side note: I have added a <tt>test</tt> target to the NetBSD Makefiles, so now it's possible to just do <tt>make test</tt> within any subdirectory of <tt>src/tests/</tt> and get what you expect.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5879847356596187843?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: EuroBSDCon 2010 - Call for Papers

Wed, 16 Jun 2010 23:56:00 +0000

From some mails I see: ``EuroBSDCon 2010 - Call for Papers

9th European BSD Conference
October 8 - 10, 2010
Karlsruhe, Germany
http://2010.eurobsdcon.org/

Introduction

The European BSD Community will meet again this year for the ninth conference in the EuroBSDCon series. This is a great opportunity to present new ideas to the community, inform your fellow BSD enthusiasts about the newest developments and work for the continued success of your favorite operating system. The two day conference program (October 9 - 10) will be preceeded by a tutorial day (Oct 8). Call for Papers

We are inviting contributions on all areas relating to the BSD family of operating systems, e.g. applications, architecture, implementation, administration and security of *BSD operating systems ranging from embedded systems to mainframes. Investigations on economic aspects regarding the operation of BSD systems are also welcome.

Prospective authors of contributions to the technical program are requested to submit an abstract via http://2010.eurobsdcon.org/. Presentations should last about 40 minutes including time for questions from the audience. Authors of accepted submissions should provide a full paper for publication in the conference proceedings and give permission to the organizers to publish the results in the printed proceedings and on the conference web site at www.eurobsdcon.org.

Call for Tutorial Proposals

Selected tutorials will be offered on the day before the conference. If you are interested in presenting a tutorial, please submit your suggestion on the conference website using the same mechanism as for submitting a paper. Please indicate if this would be a half- or full-day tutorial.

Sponsorship Opportunities

We are seeking companies or institutions to sponsor various elements of the conference in order to keep delegate fees as low as possible. Sponsorship opportunities include: paying for a speaker's travel or accommodation; providing bursaries for delegates who cannot pay the conference fee themselves; sponsoring the social event or the printing of proceedings. Please see the conference website for details.

Important Dates

Final abstract deadline: July 6th 2010
Final tutorial deadline: July 6th
Final papers due: September 1st
Tutorial day: October 8th
Conference: October 9 - 10

For more, see http://2010.eurobsdcon.org/''

Hubertf's NetBSD Blog: Hiding other users' processes

Sat, 05 Jun 2010 18:42:00 +0000

Thus it was asked on #NetBSD:

<batence> I wanna set the top command work only for users process, not
                for all system
<batence> in freebsd command is sysctl security.bsd.see_other_uids=0/1
<batence> but I dunno for netbsd
<batence> eg I don't want users see other uids
<batence> only which they owned

Looking at the output of "sysctl -a" didn'r show anything obvious, but recalling the topic and with some digging, there actually is a sysctl switch for that in NetBSD: security.models.bsd44.curtain=1

Here's an example top(1) output with the default setting (0). My username is "feyrer", note that besides my processes, other users' processes are shown as well:

load averages:  0.02,  0.01,  0.00;               up 11+15:08:30                           18:38:56
24 processes: 23 sleeping, 1 on CPU
CPU states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% idle
Memory: 71M Act, 51M Inact, 552K Wired, 5560K Exec, 110M File, 27M Free
Swap: 512M Total, 335M Used, 178M Free

  PID USERNAME PRI NICE   SIZE   RES STATE      TIME   WCPU    CPU COMMAND
    0 root     126    0     0K   16M pgdaemon   5:41  0.00%  0.00% [system]
  492 root      85    0  4792K  608K kqueue     0:06  0.00%  0.00% master
  113 root      85    0  2908K  860K select     0:05  0.00%  0.00% dhclient
  535 root      85    0  2900K  556K nanoslp    0:05  0.00%  0.00% cron
  155 root      85    0  2932K  548K kqueue     0:05  0.00%  0.00% syslogd
  496 postfix   85    0  4792K  888K kqueue     0:01  0.00%  0.00% qmgr
 4409 feyrer    43    0  2984K 1240K CPU        0:00  0.00%  0.00% top
 1197 root      85    0  8640K 3692K netio      0:00  0.00%  0.00% sshd
24830 root      85    0  8640K 3692K netio      0:00  0.00%  0.00% sshd
 6949 feyrer    85    0  8640K 2828K select     0:00  0.00%  0.00% sshd
28093 feyrer    85    0  8640K 2828K select     0:00  0.00%  0.00% sshd
12391 feyrer    85    0  2132K 1876K pause      0:00  0.00%  0.00% tcsh
25579 feyrer    85    0  2132K 1876K pause      0:00  0.00%  0.00% tcsh
 5773 postfix   85    0  4792K 1868K kqueue     0:00  0.00%  0.00% pickup
 1929 root      85    0  2128K 1828K ttyraw     0:00  0.00%  0.00% tcsh
29212 root      85    0  2972K 1164K kqueue     0:00  0.00%  0.00% inetd
25972 root      85    0  2824K 1076K pause      0:00  0.00%  0.00% ksh

Likewise, I see a number of processes in ps(1):

% ps -aux | wc -l
      26

Now let's change the sysctl:

# sysctl -d security.models.bsd44.curtain
security.models.bsd44.curtain: Curtain information about objects to users not owning them.
# sysctl -w security.models.bsd44.curtain=1
security.models.bsd44.curtain: 0 -> 1

After this, the top(1) output looks like this:

load averages:  0.02,  0.01,  0.00;               up 11+15:08:45                           18:39:11
5 processes: 4 sleeping, 1 on CPU
CPU states:  0.0% user,  0.0% nice,  0.2% system,  0.0% interrupt, 99.8% idle
Memory: 71M Act, 51M Inact, 552K Wired, 5416K Exec, 110M File, 28M Free
Swap: 512M Total, 335M Used, 178M Free

  PID USERNAME PRI NICE   SIZE   RES STATE      TIME   WCPU    CPU COMMAND
 4409 feyrer    43    0  2984K 1240K CPU        0:00  0.00%  0.00% top
28093 feyrer    85    0  8640K 2828K select     0:00  0.00%  0.00% sshd
 6949 feyrer    85    0  8640K 2828K select     0:00  0.00%  0.00% sshd
12391 feyrer    85    0  2132K 1876K pause      0:00  0.00%  0.00% tcsh
25579 feyrer    85    0  2132K 1876K pause      0:00  0.00%  0.00% tcsh

This reduced set of processes is also shown in ps(1):

% ps -aux | wc -l
       7

In other words, only my processes are displayed. (If you wonder about the difference between the 7 processes shown in top and the seven ps(1)-lines: the latter includes a heading).

Note that this "filtering" does not apply to the root user, i.e. he can still see all processes.

Hubertf's NetBSD Blog: Videos: Booting NetBSD [Update #3]

Mon, 31 May 2010 21:15:00 +0000

Jun Ebihara wrote me that there are a bunch of videos on YouTube, showing NetBSD boot on various machines:

Update: Links added for those not seeing the embedded videos (which I've seen happens via at least two RSS aggregators)

Update #2: Added booting NetBSD/hpcsh 5.1_RC2 from Windows CE on PERSONA, also submitted by Jun Ebihara. Thanks a lot!

Update #3: Added booting NetBSD/dreamcast 5.1_RC2 with IDE HDD and NE2000 NIC

Hubertf's NetBSD Blog: NetBSD 5.1_RC2 binaries available for download

Fri, 28 May 2010 14:30:00 +0000

Soren Jacobsen writes on netbsd-announce: ``The second release candidate of NetBSD 5.1 is now available for download at:
http://ftp.NetBSD.org/pub/NetBSD/NetBSD-5.1_RC2/
Those of you who prefer to build from source can continue to follow the netbsd-5 branch, but the netbsd-5-1-RC2 tag is available as well.
See src/doc/CHANGES-5.1 for the list of changes from RC1 to RC2.

Please help us test this and any upcoming release candidates as much as possible. Remember, any feedback is good feedback. We'd love to hear from you, whether you've got a complaint or a compliment. ''

Hubertf's NetBSD Blog: Source-changes ketchup Dec'09 - May'10 [Updated]

Fri, 28 May 2010 02:22:00 +0000

Here's what I have in my source-changes folder as interesting changes between Dec '09 and May '10. YMMV:

NetBSD/cats now uses X.org
NetBSD/sparc64 now runs a MULTIPROCESSOR kernel by default
NetBSD/evbsh3 now also supports the following boards: AlphaNet MS104-SH4, TAC T-SH7706LAN Ver.3, TAC T-SH7706LSR Ver.1
NetBSD/hpcarm now supports the Sharp W-ZERO3 series
wc(1) can print the longest line length now with -L
cdbr(3), cdbw(3) implement constand database reader/writer routines. this shrinks the services(5) database from 2.1MB to 307kB
/dev/{mem,kmem,zero,null} implementations are unified in machine independent code on the rmind-uvmplock branch
Many kernel systems were prepared to be built either into a monolithic kernel, or loaded as module at runtime. This includes verbose listing of PCI devices, Berkeley Packet Filters (bpf), loading modules only after the root filesystem is mounted, others.
New drivers:
- acpiwmibus: a pseudo-bus to which child Microsoft Windows Management Instrumentation (WMI, a subset of ACPI) devices attach
- u2g: split into parts: u3ginit attaches to those devices that only come as a umass device in the default configuration and forces them to reinitialize in 3D mode and detach. The u3g part attaches to individual interfaces for the 3G functionality, leaving the umass interface(s) for that driver. With this change I can use the MMC card in my Huawey stick (as well as the integrated windows driver CD, which of course is pretty useless) and the 3G modem at the same time.
- cas: Driver for Sun Cassini/Cassini+ (GigaSwift) Ethernet (also known as National Semiconductor Saturn)
- aibs: replaces aiboost for the ASUSTeK AI Booster hardware monitor
Software imports include NetPGP 3.99.2 (20100507 version), dhcpcd-5.2.2, BIND 9.7.0rc1, libelf from FreeBSD 8.0-RELEASE, tzcode2009k, pkg_install-20100220, mdocml-1.9.23, libarchive-2.8.2, ACPICA 20100121, bozohttpd-20100512

Hubertf's NetBSD Blog: NetBSD ketchup - news from my mailbox

Fri, 28 May 2010 01:31:00 +0000

Here's another bunch of NetBSD-related news that has been lingering in my inbox for far too long:

Izumi Tsutsui's NetBSD/cobalt restore CD is available based on NetBSD versions 5.0.2 5.1_RC2. See the for information on what it is and how to use it.
A negative symbol lookup cache was added to NetBSD's loader for shared libraries and shared objects, ld.so_elf, by Roy Marples: ``I've been researching why Evolution from GNOME takes over 5 minutes to load on my quad core amd64 beast. It boils down to dlsym looking for a symbol that does not exist directly and as such examining every needed library. However, the current implementation does not remember what libraries it as already checked. Normally this isn't a problem, but with the way Evolution is built the search chain is massive. [...]
With this patch, Evolution (without the patches to and a glib I added to pkgsrc a few days ago) loads in under 2 seconds (5 seconds with initial disk thrashing). ''
The NetBSD Logo is available in many variants, but a new variant was submitted via www@ these days by "Tim" - which is actually plain HTML, no image:
⚑ NetBSD Powered!
SafeNet's ProtectDrive is ``a full disk encryption solution that encrypts the entire hard drive of laptops, workstations and servers, as well as USB flash drives, to protect data in the case of the theft or loss of a hardware device.''
How do you implement such preboot authentication and harddisk encryption software, esp. if you want to provide thinks like LDAP integration for the user/key handling and two-factor authentication? Little is known, but rumors say the 32bit version of the software is based on NetBSD, as is backed by this worker bio info: ``Duties: Working on pre-boot restricted environment with loads before operation system and implemented on NetBSD. Ported and optimized the KDrive X server to NetBSD. Developed and implemented user secure authentication interface with smart card support.
Environment and tools : NetBSD (3.0), C/C++, FLTK''
A german-language introduction of pkgsrc on OpenSolaris was given by Michael 'kvedulv' Moll at the Munich OpenSolaris User Group back in march. Slides and a video are available.
Running NetBSD on an ~~Oracle~~ Sun Fire X4140 Server? Check out this posting by Ignatios Souvatzis for the full dmesg pr0n of this machine with 12 CPU cores and 32GB RAM!
Are you still looking for a nice small ARM-based board to start hacking on NetBSD/arm? The http://www.friendlyarm.net/products/mini2440 may be a good start, esp. after Paul Fleischer is reaching completion of NetBSD support for the board. Citing from his mail to port-arm:
``I have now fairly good (i.e., it works for me) support for the MINI2440 on NetBSD with support for the following:
- S3C2440 UART
- DM9000 (MAC+PHY)
- S3C2440 SD Controller
- S3C2440 DMA Controller
- S3C2440 IIS Controller
- FriendlyArm 3,5" LCD Display
- S3C2440 USB Host Controller (OHCI)
- S3C2440 Touch Screen
- UDA1341TS audio codec

Currently, support for three things on the S3C2440 are missing:
- S3C2440 NAND Controller
- S3C2440 USB Device Controller
- S3C2440 RTC

I've also created a stage2 bootloader for use with u-boot, which ensures that the value of bootargs is passed to the NetBSD kernel. At this point I have only tested the code with the 64Mb version of the FriendlyArm MINI2440.
All the code is available in a Git repository[1] and is based on the netbsd-5 code base. Progress can be followed on my webpage[2]. ''
While talking about NetBSD on cool hardware: How about NetBSD/hpcarm on WILLCOM | W-ZERO3 (WS004SH) mobile devices? Here is a screenshot of Ebihara-san's WS011SH with CCW screen, and there is also a video "booting NetBSD/hpcarm on WILLCOM | W-ZERO3(WS004SH)" posted on YouTube:

For more details, see Izumi Tsutsui's posting on port-hpcarm.

Hubertf's NetBSD Blog: Apple Magic Mouse driver

Sun, 16 May 2010 22:33:00 +0000

NetBSD's Bluetooth hacker #1, Iain Hibbert, wrote on tech-kern: ``I wrote a driver for the Apple Magic Mouse, as the protocol was mostly decoded by a Linux developer, and Somebody was kind enough to send me one. [...]
The mouse itself is a wireless Bluetooth mouse and operates with the USB HID protocol much like other mice, but it doesn't provide a proper descriptor and requires features to be activated and special interpretations of the touch surface reports, so doesn't fit exactly into our HID framework, which configures independent sub-devices to report id's from the descriptor.

The driver interprets the touch reports to allow emulation of a middle mouse button (for mulitple firm touches detected), and horizontal and vertical scroll actions (for touches moving over a certain distance). It works well on NetBSD-current and NetBSD-5 and the mouse is pretty slick. '' See Iain's posting for more details.

The Julipedia (Blog): Trac installation for ATF

Julio Merino — Mon, 10 May 2010 18:48:06 +0000

During the past few months, I've got into the habit of using a bug tracker to organize my tasks at the work place. People assign tickets to me to get things done and I also create and self-assign tickets to myself to keep them as a reminder of the mini-projects to be accomplished. Sincerely, this approach works very well for me and keeps me focused.<br /><br />Since then, I've been wishing to have a similar system set up for ATF. Yeah, we could use the <a href="http://www.netbsd.org/Gnats/">Gnats</a> installation provided by NetBSD... but I hate this issue tracking system. It's ancient, ugly, and I do really want a web interface to manage my tickets through.<br /><br />So, this weekend, I finally took some time and set up a Trac installation for ATF to provide a decent bug/task tracking system. The whole Apache plus Trac setup was more complex than I imagined, but I do hope that the results will pay off :-)<br /><br />Take a look at the <a href="http://www.NetBSD.org/~jmmv/atf/news.html#20100509-trac-available">official announcement</a> for more details!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-7270221803608994472?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Ads gone

Julio Merino — Mon, 10 May 2010 09:48:06 +0000

Almost a year ago, I decided to <a href="http://blog.julipedia.org/2009/06/trying-adsense.html">give a try to AdSense</a>. And, so far, the "earnings" have been ~30 EUR which I cannot even cash. Given this and how ugly and disturbing the ads look on the front page, I have disabled them. (I think the ads have gotten much worse over time... but as I do not pay attention to the front page, I didn't see them.) Thanks to Roman Valls for pointing this out!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-3049034875819271527?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): ATF 0.8 imported into NetBSD

Julio Merino — Sat, 08 May 2010 08:48:05 +0000

Finished <a href="http://mail-index.netbsd.org/source-changes/2010/05/08/msg009703.html">importing ATF 0.8 into the NetBSD source tree</a>. Wow, the CVS import plus merge was much easier than I expected.<div><br /></div><div>Note that, while the NetBSD test suite should continue to work as usual, there are some backwards incompatible changes in the command line interface of test programs. If you are used to run them by hand, expect different results. Please read the <a href="http://www.NetBSD.org/~jmmv/atf/news.html#20100507-atf-0-8-released">release news</a> for details.<br /><div><br /></div><div>Now let's wait for complaints about broken builds! And enjoy this new release in your NetBSD-current system!</div></div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5123310572457338623?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Announcing ATF 0.8

Julio Merino — Fri, 07 May 2010 15:48:09 +0000

Looks like today is a release day. I've just pushed <a href="http://www.NetBSD.org/%7Ejmmv/atf/news.html#20100507-atf-0-8-released">ATF 0.8</a> out in the wild and will proceed to import it into pkgsrc and NetBSD later. Refer to the release announcement for details. This is an exciting release! You have been warned ;-)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5230262632323813624?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Announcing etcutils 0.1

Julio Merino — Fri, 07 May 2010 09:48:06 +0000

During past week, I worked on a new package called etcutils. It provides a (reduced) tool-set to programmatically manage files in <tt>/etc</tt> and is specially designed to allow pkgsrc to update <tt>/etc/shells</tt> and <tt>/etc/services</tt> in a more consistent way.<br /><br />I'm happy to say that the 0.1 release is now ready! Go to the <a href="http://www.netbsd.org/%7Ejmmv/etcutils/">etcutils web page</a> for details. (I know that if you are a Linux user, you probably don't care about this because your distribution most likely already provides something similar... albeit more complex.)<br /><br />I'll now proceed to import this new package into pkgsrc as <tt>sysutils/etcutils</tt>. Later on (most likely not today), the following should happen: rework pkginstall to use the new shells(8) utility to update <tt>/etc/shells</tt>, add a new feature to pkgsrc to abstract the updates to <tt>/etc/services</tt>, and swipe through pkgsrc to make all packages touching this file use the new frameworky option. (Hey FAM, that includes you!)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-6570217818735603714?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: Silencing the boot process

Sat, 01 May 2010 10:52:00 +0000

NetBSD-current is able to boot the userland silently with "boot -z" for quite some time now, thanks to Alan Barrett. Those changes were never ported back to the netbsd-5 branch so far, but I'm in the process to change this now. Here's a preview:

Of course there are still many places left in the kernel that don't honor the boot flags (i.e. that use printf(9) instead of aprint_normal(9) and friends), but this will change over time - I hope :-).

Hubertf's NetBSD Blog: NetBSD-current build status

Thu, 29 Apr 2010 00:54:00 +0000

Have you ever wondered how stable a -current build is at times? Here is an interesting page to give an overview on the success of NetBSD-current builds from the past few days (and much further back):

From the webpage: ``This web page visualizes the state of the NetBSD-current build by plotting the number of lines in the build log from build.sh as a function of time, and coloring the points red or green depending on whether the build succeeded or failed. Hopefully, this can help give some insight into the frequency and duration of build failures.

The plots were constructed using an automated procedure that builds NetBSD-current periodically (roughly twice a day) using make.sh -m i386 release, looks for any changes in exit status or any substantial changes in the size of the build log, and then pinpoints the times at which the changes occurred using binary search. The graph may not contain every build failure, because the algorithm used can miss cases where the build is broken and then fixed again between two consecutive periodic builds. The converse case, when the build is fixed and then broken again between two consecutive periodic builds, is usually detected because the size of the build log usually changes in this case.

Most of the builds in the graph are cross-builds using a 64-bit Linux host, while some of the older ones are native NetBSD builds.

If your browser supports SVG, you can use the SVG version. ''

Thanks to Andreas Gustafsson for providing this service!

The Julipedia (Blog): Forget about test(1)'s == operator

Julio Merino — Sat, 24 Apr 2010 21:48:06 +0000

Some implementations of test(1), in an attempt to be smart, provide non-standard operators such as <tt>==</tt>. Please forget about those: they make your scripts non-portable and a pain to use in other systems. Why? Because, due to the way the shell works, failures in calls to test(1) will often just result in an error message (which may not be seen due to other output) and the script will happily continue running even if it missed to perform some important operation.<br /><br />So... just use the standard equality operators:<br /><ul><li><tt>=</tt> for string equality comparison.</li><li><tt>-eq</tt> for numeric equality comparison.</li></ul>Note that whenever I refer to test(1), I'm also talking about the <tt>[ ... ]</tt> construction in conditionals.<br /><br />Also, please note that this also affects configure scripts, and the problem in these appears much more commonly than in other scripts!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-6205418080955020541?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Always define an else clause for portability #ifdefs

Julio Merino — Thu, 22 Apr 2010 14:48:06 +0000

If you use <tt>#ifdef</tt> conditionals in your code to check for portability features, be sure to always define a catch-all else clause that actually does something, even if this something is to error out.<br /><br />Consider the following code snippet, quoted from gamin's <tt>tests/testing.c</tt> file:<pre>if (arg != NULL) {<br />#ifdef HAVE_SETENV<br /> setenv("GAM_CLIENT_ID", arg, 1);<br />#elif HAVE_PUTENV<br /> char *client_id = malloc (strlen (arg) + sizeof "GAM_CLIENT_ID=");<br /> if (client_id)<br /> {<br /> strcpy (client_id, "GAM_CLIENT_ID=");<br /> strcat (client_id, arg);<br /> putenv (client_id);<br /> }<br />#endif /* HAVE_SETENV */<br />}<br />ret = FAMOpen(&amp;(testState.fc));</pre>The <tt>FAMOpen</tt> method queries the <tt>GAM_CLIENT_ID</tt> environment variable to set up the connections parameters to the FAM server. If the variable is not defined, the connection will still work, even though it will use some default internal value. In the test code above, the variable is explicitly set to let the tests use a separate server instance.<br /><br />Now, did you notice that we have to conditional branches? One for <tt>setenv</tt> and one for <tt>putenv</tt>? It seems reasonable to assume that one or the other must be present on any Unix system. Unfortunately, this is flawed:<br /><ul><li>What happens if the code forgets to include <tt>config.h</tt>?</li><li>What happens if the configure script fails to detect <i>both</i> <tt>setenv</tt> and <tt>putenv</tt>? This is not that uncommon, given how some configure scripts are written.</li><li>What happens if neither <tt>setenv</tt> nor <tt>putenv</tt> are available?<br /></li></ul>The answer to the three questions is: in the above code snippet, the code <tt>builds just fine</tt> but will misbehave at run time: neither <tt>HAVE_SETENV</tt> nor <tt>HAVE_PUTENV</tt> are defined, so the code will not be able to define the required environment variable. However, <tt>FAMOpen</tt> will later be called and it will not behave as expected because the variable has not been set.<br /><br />Note that this code snippet is just an example. I have seen many more instances of this exact same problem with worse consequences than the above. Read: they were not part of the test code, but just part of the regular code path.<br /><br />So how do you implement the above in a saner way? You have two alternatives:<br /><ul><li>Add an <tt>#else</tt> clause that contains a fallback implementation. In the case above, we could, for example, prefer to use <tt>setenv</tt> if present because it has a nicer interface, and fall back to <tt>putenv</tt> if not found.<br />This has a disadvantage though: if you forget to include <tt>config.h</tt> or the configure script cannot correctly detect one of the possible implementations (even when present), you will always use the fallback implementation.</li><li>Keep each possible implementation correctly protected by a conditional, but add a <tt>#else</tt> clause that raises an error at <i>build time</i>. This will make sure that you never forget to define at least one of the portability macros for any reason. This is the preferred approach.</li></ul>Following the second suggestion above, the code would get the following structure: <pre>#if defined(HAVE_SETENV)<br />setenv(...);<br />#elif defined(HAVE_PUTENV)<br />putenv(...);<br />#else<br /># error "Don't know how to set environment variables."<br />#endif</pre>With this code, we can be sure that the code will not build if none of the possible implementations are selected. We can later proceed to investigate why that happened.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1519509019468318358?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Where does Gnome use file monitoring?

Julio Merino — Wed, 21 Apr 2010 08:48:06 +0000

One of <a href="http://blog.julipedia.org/2010/04/new-gio-fam-package.html?showComment=1271802660697#c3047587885904955995">my readers</a> of my <a href="http://blog.julipedia.org/2010/04/new-gio-fam-package.html">post yesterday</a>, wonders where Gnome uses the file monitoring APIs. Well, the answer is: <span style="font-style: italic;">everywhere</span>.<br /><br />Here are some examples:<br /><ul><li>Nautilus monitors all open folders so that it can update their contents whenever the underlying file store changes. Say you are viewing the Documents folder and you save a new Document from within OpenOffice into that folder. You definitely want Nautilus to show it immediately, without having to manually hit Refresh from the menu.</li><li>The trash applet monitors the trash folders and updates its icon from empty to full whenever one of these folders ceases to be non-empty.</li><li>The panel monitors the applications directory to notice when new applications get installed. This allows it to update the Applications menu immediately as soon as a new program gets installed into the system.</li><li>The GTK file open/save dialogs monitor the directory they are viewing for the same reason as Nautilus. (Actually, I'm unsure about this point. My Linux installation is an old Ubuntu 8.04 LTS that does not have GIO, so I can't verify in recent ones. However, this makes perfect sense and if not implemented, it should be!)</li><li>The background switcher control panel monitors the folders containing images to be able to show new installed backgrounds. (I'm not sure about this either. It doesn't happen in my Linux installation, but it also makes sense.)</li><li>Media players such as Rhythmbox and Banshee allow you to point them to a folder containing music and have an option to automatically add music to the library as soon as it pops up in such folder.</li><li>Potentially, any document editor, picture viewer, etc. monitors the documents opened in them so that these applications can notice external notifications to those files. This is useful to prevent overwriting a file with an out-dated in-memory copy. For example: you are taking some notes with GEdit. On a separate terminal window, you quickly edit the notes file with Vim to add a new note. When you go back to GEdit, you want the editor to tell you that the file has changed out of its control and offer you a choice: e.g. reload or ignore?<br /></li></ul>And many, many more other situations that I'm surely missing...<br /><br />As you can see, it is fairly important to get the file monitoring subsystem working flawlessly. Otherwise, all these tiny details don't work and the end user experience is undermined.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-8725416766704267720?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): New gio-fam package

Julio Merino — Tue, 20 Apr 2010 18:48:06 +0000

As briefly outlined in the <a href="http://blog.julipedia.org/2010/04/file-system-monitoring-gnome-and-netbsd.html">previous post</a>, new versions of Glib provide GIO, a library that intends to be a low-level file system API on top of the POSIX interface. This library provides an interface to asynchronously wait for file system change notifications including the creation, deletion and modification of files.<br /><br />The monitoring functionality in GIO is modular: it is backed by different loadable plugins that implement OS-specific functionality. In particular, GIO uses an inotify module in Linux and a <a href="http://oss.sgi.com/projects/fam/">FAM</a> module everywhere else.<br /><br />Up until now, the <tt>devel/glib2</tt> package in pkgsrc provided a build-time option to specify whether to build the GIO FAM plugin or not. Given that this plugin is built as a shared object that is loaded dynamically at run-time, having a build-time option for this is clearly wrong: it gives no choice to those relying on binary packages (e.g. end/new users). Furthermore, it adds a dependency on the ugly-FAM at the very bottom of the huge Gnome dependency chain. (As already stated, FAM is outdated and hard to set up.)<br /><br />So, based on this, I've just removed all FAM support from <tt>devel/glib2</tt> altogether and packaged its loadable module as <tt>sysutils/gio-fam</tt>.<br /><br />Now waiting for a clean rebuild of the Gnome packages to see if the desktop now works on my machine by avoiding FAM/Gamin.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-4070965403269265546?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): File system monitoring, Gnome and NetBSD

Julio Merino — Tue, 20 Apr 2010 14:48:07 +0000

A few days ago, I decided to start using NetBSD, as well as Gnome on NetBSD once again, mostly because the lack of their use makes my skills feel rusty in many different areas. While NetBSD has surprised me in a good way (I am running it on a Macbook Pro and things like wireless and DRI work), Gnome has not. There are tons of broken things that prevent a smooth user experience.<br /><br />One of these broken things is the monitoring of changes in the file system. Actually, this has never worked 100%. But what is this and why does it matter, you ask? Well, file system monitoring is an internal component of the Gnome infrastructure that allows the desktop to receive notifications when files or directories change. This way, if, say, you are viewing the Downloads folder in Nautilus and you start downloading a file from Epiphany into that folder, Nautilus will realize the new file and show it immediately without requiring a manual refresh.<br /><br />How to monitor the file system depends on the operating system. There are basically two approaches: polling and asynchronous notifications. Polling is suboptimal because the notifications are usually delayed. Asynchronous notifications are tied to the operating system: Linux provides <a href="http://en.wikipedia.org/wiki/Inotify">inotify</a>, NetBSD provides <a href="http://en.wikipedia.org/wiki/Kqueue">kqueue</a> and other systems provide their own APIs.<span style="display: block;" id="formatbar_Buttons"><span class="on down" style="display: block;" id="formatbar_CreateLink" title="Link" onmouseover="ButtonHoverOn(this);" onmouseout="ButtonHoverOff(this);" onmouseup="" onmousedown="CheckFormatting(event);FormatbarButton('richeditorframe', this, 8);ButtonMouseDown(this);"><img src="http://www.blogger.com/img/blank.gif" alt="Link" class="gl_link" border="0" /></span></span><br />In the past, Gnome monitored the file system by a combination of <a href="http://oss.sgi.com/projects/fam/">FAM</a>, a system-level service that provides an API to file system monitoring, and GNOME VFS, a high-level layer that hides the interaction with FAM. This approach was good in spirit (client/server separation) but didn't work well:<br /><ul><li>FAM is abandoned.<br /></li><li>Does not support kqueue out of the box.</li><li>FAM runs as root.<br /></li><li>FAM is too hard to set up: it requires rpcbind, an addition to <tt>/etc/services</tt>, a sysctl tweak, and the configuration of a system-level daemon.</li></ul>To solve some of these problems, a drop-in replacement for FAM was started. <a href="http://people.gnome.org/%7Eveillard/gamin/">Gamin</a>, as it is known, still does not fix everything:<br /><ul><li>Gamin is abandoned.</li><li>Supports kqueue out of the box, but does not work very well.</li><li>Actually, Gamin itself does not work. Running the tests provided by the distfile in a modern Linux system results in several test failures.<br /></li></ul>Anyway. Did you notice the <span style="font-style: italic;">abandoned</span> pattern above? This is important: in the new world order, Gnome does not use FAM any more.<br /><br />The new structure to monitor files is: the low-level glib library provides the gio module, which has some file system monitoring APIs. The GVFS module provides higher level abstractions to file system management, and relies on gio for file system monitoring. There is no more GNOME VFS any more.<br /><br />The <span style="font-style: italic;">key</span> point is: gio uses inotify directly; no abstraction layers in between. FAM support is still there for platforms without inotify, but as it is not used in Linux any more, it rots. Linux developers will never experience what it is to have a system that needs to use FAM to get this functionality to work.<br /><br />At last, let's look at the status of all this in NetBSD:<br /><ul><li>The FAM package was <a href="http://blog.julipedia.org/2004/10/fam-and-kqueue.html">patched to support kqueue</a>. Although this kinda works, it is not perfect. Also, as mentioned above, FAM is, I'd say, the package with the hardest installation procedure of the whole Gnome platform.<br /></li><li>The Gamin packages are nicer than the FAM package regarding their configuration. However, when using Gamin instead of FAM, all sorts of bugs appear in Gnome (it actually gets stuck during startup for me). The breakage of the unit tests does not provide any confidence, and the fact that Gamin is abandoned, the idea of fixing it doesn't make me thrive.</li><li>The glib2 package <span style="font-style: italic;">depends</span> on FAM. This is ugly; really ugly. I had to shout WTF when I saw this, seriously.</li><li>Seeing the direction gio/gvfs take, it is obvious that things can only get worse in the future.<br /></li></ul>If time permits, I'm planning to work on improving this whole situation. Ideas include:<br /><ul><li>Splitting the FAM gio module out of the glib2 package. Ideally, this would happen upstream.</li><li>Implement a gio backend for kqueue.</li><li>Check if the core packages still using gnome-vfs have a more recent version that uses gvfs instead and, if so, update them.<br /></li></ul>Can't promise you anything other than, if I get to work on it, I will keep you posted!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5347499839576127054?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: Unixes of the world unite!

Tue, 13 Apr 2010 23:42:00 +0000

I've stumbled about this presentation by Unix (co)pioneer Robert Pike, which he gave in 2001. Some hilights:

``What is Unix^TM?
[...] Those operating systems derived from or inspired by the Research Unix systems of the 1970.
Includes [...] NetBSD [...] and others.''
``1. What is the best thing about Unix?
A: The community.
2. What is the worst thing about Unix? A: That there are so many communities.''

Read the whole presentation for an overview of the things that Unix got right and wrong. Definitely worth reading!

Hubertf's NetBSD Blog: NetBSD and Google Summer of Code - Send in your proposals!

Tue, 06 Apr 2010 21:31:00 +0000

Just a friendly reminder to interested students who want to participate in this year's Google Summer of Code: You have only three days left to send in your proposal! Student application deadline is April 9th, 19:00 UTC.

See our list of suggested SoC projects if you need inspiration (other project suggestions are welcome!), and please have a look at our proposal HowTo for a bunch of questions we want to have answered so that we can properly judge your proposal. Get going!

The Julipedia (Blog): NetBSD in Google Summer of Code 2010

Julio Merino — Fri, 19 Mar 2010 09:48:06 +0000

For the 6th year in a row, <a href="http://www.netbsd.org/">NetBSD</a> is a mentoring organization for <a href="http://code.google.com/soc/">Google Summer of Code</a> 2010!<br /><br />If you are a bright student willing to develop full-time for an open source project during this coming summer, consider applying with us! You will have a chance to work with very smart people and, most likely, in the area that you are most passionate about. NetBSD, being an operating system project, has offers for project ideas at all levels: from the kernel to the packaging system, passing by drivers, networking tools, user-space utilities, the system installer, automation tools and more!<br /><br />I would like to point you at the 3 project proposals I'm willing to directly mentor:<br /><ul><li><a href="http://www.netbsd.org/contrib/soc-projects.html#optimize-atf">Optimize and speed-up ATF</a>: Make the testing framework blazing fast so that running the NetBSD automated tests does not take ages on slow platforms.<br /></li><li><a href="http://www.netbsd.org/contrib/soc-projects.html#reorganize-atf">Reorganize ATF to improve modularity</a>: Refactor pieces of the testing framework so that it is easier to redistribute, has cleaner interfaces and is easier to depend on from third-party projects.<br /></li><li><a href="http://www.netbsd.org/contrib/soc-projects.html#rewrite-pkg_comp">Rewrite pkg_comp with portability as a major goal</a>: Use Python to create a tool to automatically build binary packages from within a sandbox.</li></ul>If you find any of the above projects interesting, or if you have any other project proposal that you think I could mentor, do not hesitate to <a href="mailto:jmmv%20at%20netbsd%20dot%20org">contact me</a>. Feel free to send me a draft of <a href="http://www.netbsd.org/contrib/soc-application.html">your application</a>, together with a bit of information about you, so that we can discuss your proposal and make sure it gets selected!<br /><br />Or, if none of the projects above interests you, please do check out the <a href="http://www.netbsd.org/contrib/soc-projects.html">full list of NetBSD project proposals</a>. I'm sure you will find something that suits your interests :-)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1228455004173703548?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: NetBSD is part of Google's Summer of Code 2010

Thu, 18 Mar 2010 23:07:00 +0000

This year's Google Summer of Code mentoring organizations has been announced, and NetBSD is part of it, again! Possible projects are listed on the projects page, the GSoC wiki page, and esp. on the SoC-projects page.

Prospective students that are interested in working on a project are recommended to have a look at the Project Application / Proposal HowTo!

FreeBSD Diary: Monitoring your HDD using SMART and Nagios

Sat, 13 Mar 2010 00:48:08 +0000

Checking your HDD!

The Julipedia (Blog): New version of the monotone-server package in pkgsrc

Julio Merino — Fri, 12 Mar 2010 09:48:06 +0000

Wow, it has been a long time... 5 years ago, I created the <a href="http://cvsweb.netbsd.org/bsdweb.cgi/pkgsrc/devel/monotone-server/?only_with_tag=MAIN">monotone-server</a> package in pkgsrc, a package that provided an interactive script to set up a <a href="http://monotone.ca/">monotone</a> server from scratch with, what I though, minimal hassle.<br /><br />My package did the job just fine, but past year I was blown away by the simplicity of the same package in <a href="http://www.fedoraproject.org/">Fedora</a>: their <tt>init.d</tt> script provides a set of extra commands to initialize the server before starting it up, and that is it. No need to mess with a separate interactive script; no need to create and memorize passphrases that you will never use; and, what's more, all integrated in the only single place that makes sense: in the init.d "service management" script.<br /><br />It has been a while since I became jealous of their approach, but I've finally got to it: I've spent the last few days rewriting the monotone-server package in pkgsrc and came up with a similar scheme. And this <a href="http://mail-index.netbsd.org/pkgsrc-changes/2010/03/12/msg038844.html">new package</a> just made its way <a href="http://mail-index.netbsd.org/pkgsrc-changes/2010/03/12/msg038844.html"></a>to pkgsrc-HEAD! The new package comes with what I think is a detailed manual page that explains how to configure the server from scratch. Take a look and, if you find any mistakes, inconsistencies or improvements to be done, let me know!<br /><br />In the meantime, I will log into my home server, rebuild the updated package and put it in production :-)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1599070089952057305?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Introducing the ATF nofork branch

Julio Merino — Sat, 06 Mar 2010 20:48:06 +0000

Despite my time for free software being virtually zero these days, I have managed to implement a prototype of what ATF would look like if it didn't implement forking and isolation in test programs. This feature has been often requested by users to simplify their life when debugging test cases.<div><div><br /></div><div>I shouldn't repeat everything I posted on the atf-devel mailing list regarding this announcement, so please <a href="http://mail-index.netbsd.org/atf-devel/2010/03/05/msg000070.html">refer to that email for details</a>. But I must say that the results look promising: the overall code of ATF is much simpler and also faster. (An execution I just tried cuts the run time of the ATF test suite from 1m 41s to 1m 16s.) Expect more simplifications and speed-ups!</div></div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1501363386005733392?l=blog.julipedia.org' alt='' /></div></content>

Hubertf's NetBSD Blog: Google Summer of Code 2010 NetBSD swcryptX Project Suggestion (Updated #2)

Mon, 22 Feb 2010 22:57:00 +0000

Please see the update below before applying for this GSoC project!

I've been thinking of a neat-o project for this year's Google Summer of Code:

Abstract:

Overview of operation

The opencrypto(9) framework exists to coordinate hardware acceleration in NetBSD. Applications of the framework can be inside the kernel like the FAST_IPSEC IPsec implementation, or in userland like OpenSSL with the "cryptodev" engine. Crypto drivers can be realized in software or in hardware. Hardware drivers can be used to instruct e.g. the AMD Geode LX's AES block or a HIFN chip to perform cryptographic operations. Upon system startup, the crypto drivers at the opencrypto(9) framework, telling what operations they can perform. When an operation is required later, the framework will look which crypto device is currently not busy, and offload the operation to that device. Upon completion, the result is fed back to the application.

The following image illustrates the components and their interaction.

Limitations

Offloading the cryptographic requests involves some overhead. Data needs to be transferred to the hardware and back. On systems with a slow CPU, this overhead is relatively small compared to the operation speed of the CPU. On faster CPUs, the overhead becomes more of a burden, making the benefit of the crypto hardware negligible.

As examples, while a hifn(4) chip can provide worthwhile speedups on 500MHz and 1GHz CPUs, no performance win is experienced on a 2.4GHz CPU.

Proposal

The communication overhead involves data transfers over a PCI bus, which is of relatively low speed compared to today's modern CPUs. Preventing the data transfer is a worthwhile goal. In coordination with today's modern multi-core CPUs, using one or more CPUs solely for the purpose of crypto acceleration, a measurable improvement of crypto performance is expected. At the same time, no special hardware requirements beyond the CPU exist. This allows turning standard contemporary systems into fast crypto systems easily.

The following image illustrates the idea of interoperation between a CPU core that runs the kernel and application codes and three cores that are dedicated to crypto code.

Implementation Roadmap

This is where it gets fishy. ;) ~~The existing opencrypto(4) framework probably needs to be make MP-aware at the same time, employing proper use of NetBSD's locking framework.~~ (Already done) The existing swcrypto(4) needs to be adjusted for operation on multiple CPUs at the same time. A way to decide how many CPUs are dedicated to run swcrypto(4) instances. CPUs that run swcrypto(4) need to be taken out from the usual NetBSD CPU scheduling so that they are available exclusively for crypto.

Requirements

In no particular order:

Know how to build and install a kernel
Understanding of fine grained SMP and locking
How to use NetBSD's kernel threads, code-wise
How to interact with NetBSD's scheduler, code-wise
Tell the scheduler to pin a specific kernel thread to a specific CPU
Interaction between applications (IPsec, OpenSSL) with opencrypto(9), code-wise
Interaction of crypto providers with opencrypto(9), code-wise
Hardware! You won't be able to do this without at least two CPU cores in your machine. The more the better.
Benchmarking & a test setup for it

Project Applications

Please follow the NetBSD Project Application/Proposal HowTo if you're serious to work on this project.

If you have any questions let me know, public discussion should be led on the tech-crypto@ list.

Update: There was some discussion. In particular, my understanding of the interaction of the various layers as outlined above is not 100% accurate, and userland applications using opencrypto already seem to benefit from multiple kernel threads. In-kernel applications apparently do not, and before providing multiple crypto-servers in kernel (as suggested), work should probably done first to make sure such applications exist. Examples of this are IPsec (and the whole network stack), but also others like cgd (which AFAIU currently does not use opencrypto(9)).

The Julipedia (Blog): set -e and set -x

Julio Merino — Sun, 24 Jan 2010 22:48:06 +0000

If you write shell scripts, you definitely need to know about two nice features that can be enabled through the <tt>set</tt> builtin:<div><ul><li><tt>set -e</tt>: Enables checking of all commands. If a command exits with an error and the caller does not check such error, the script aborts immediately. Enabling this will make your scripts more robust. But don't wait until your script is "complete" to set the flag as an afterthought, because it will be a nightmare to fix the scrip to work with this feature enabled. Just write <tt>set -e</tt> as the very first line of your code; well... after the shell bang.</li><li><tt>set -x</tt>: If you are writing simple scripts that are meant to, well, script the execution of a few tasks (as opposed of being full-flown programs written in shell), set this flag to trace the execution of all commands. This will make the interpreter print each command right before it is executed, so it will aid you in knowing what is happening at any point in time.</li></ul></div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-8947551254574129862?l=blog.julipedia.org' alt='' /></div></content>

FreeBSD Diary: Problems starting a jail with ezjail

Sat, 23 Jan 2010 00:48:09 +0000

Things change. We must evolve.

FreeBSD Diary: Integrating the Tape Library with an existing Bacula installation

Tue, 19 Jan 2010 00:48:15 +0000

Adding a new tape drive isn't always simple

FreeBSD Diary: Tape Libraries & Bacula

Mon, 18 Jan 2010 00:48:10 +0000

Tape libraries can be complex to set up

The Julipedia (Blog): Installing NetBSD/macppc on a Mac Mini G4

Julio Merino — Mon, 11 Jan 2010 23:48:06 +0000

Yesterday, I spent a while installing NetBSD/macppc 5.0.1 on a Mac Mini G4. The process wasn't easy, as it involved the following steps. I'm omitting many details, as they are "common knowledge" to Mac users (or otherwise can be easily found on the net):<br /><ul><li>After booting the installer from the CD image, drop into the shell.</li><li>Use <tt>pdisk</tt> to create an <tt>Apple_HFS</tt> partition for the boot loader and two <tt>Apple_UNIX_SVR2</tt> partitions, one for the root file system and another for swap.</li><li>Run <tt>sysinst</tt> and install the system. When asked to repartition the disk, just say <i>Use existing partition sizes</i>.</li><li>Once the system is installed, drop again into the shell before rebooting.</li><li>Mount your hard disk into <tt>/mnt</tt> and chroot into it.</li><li>Fetch a copy of pkgsrc.</li><li>Install the <tt>sysutils/hfsutils</tt> package.</li><li>Use <tt>hformat</tt> to create a new HFS file system in the <tt>Apple_HFS</tt> partition we created.</li><li>Mount the installation CD.</li><li>Copy, using <tt>hcopy</tt>, the <tt>ofwboot.xcf</tt> file from the CD to the boot partition.</li><li>Reboot.</li><li>Drop into the OpenFirmware setup (Command+Option+P+R).</li><li>Set <tt>boot-device</tt> to <tt>hd:,\ofwboot.xcf</tt>.</li><li>Set <tt>boot-file</tt> to <tt>netbsd</tt>.</li><li>And here is the tricky thing to get the machine to auto-boot: Set <tt>boot-command</tt> to <tt>." hello" cr " screen" output boot</tt>, not <tt>mac-boot</tt>.</li></ul><div>I found the last command somewhere on the Internet (dunno where now), but, supposedly, a regular <tt>mac-boot</tt> should have worked. In fact, it works if you call this command from the prompt, but not during automatic boot. (It turns out to be a problem with the version of OpenFirmware I have.)</div><div><br /></div><div>Just writing down the steps in case I need them later on. Installing Debian stable was much, much easier, but the installer for testing crashes every day with a different error, so I gave up.</div><div><br /></div><div>(Oh, by the way, I did the same installation into an old PowerMac G3 and that was really painful. The machine refused to boot from any of the CDs I tried and the prebuilt kernels hang during initialization due to a bogus driver. In the end: netbooting and using custom kernels.)</div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1422463512138756573?l=blog.julipedia.org' alt='' /></div></content>

FreeBSD Diary: Upgrading PostgreSQL

Mon, 21 Dec 2009 00:48:11 +0000

Upgrading the client, then the server

The Julipedia (Blog): Processing Makefile.am with M4

Julio Merino — Mon, 16 Nov 2009 15:48:06 +0000

ATF's Makefile.am, which is a single Makefile for the whole tree, was already at the 1300 lines mark and growing. At this size, it is unmanageable, and a quick look at its contents reveals tons of repeated delicate code.<br /><br />Why so much repeated code, you ask, if the whole point of Automake is to simplify Makefiles? Automake does in fact simplify Makefile code when you define targets known by Automake, such as binaries and/or libraries. However, as soon as you start doing fancy things with documentation, building tons of small programs or messing with shell scripts, things get out of control because you are left on your own to define their targets and their necessary build logic.<br /><br />Up until now, I had just kept up with the boilerplate code... but now that I'm starting to add pretty complex rules to generate HTML and plain text documentation out of XML files, the complexity must go. And here comes my solution:<br /><br />I've just committed an experiment to <a href="http://mtn-host.prjek.net/viewmtn/atf/revision/info/aca66742e00b236d0b6dcb22ae70468736b612bc">process Makefile.am with M4</a>. I've been trying to look for prior art behind this idea and couldn't find any, so I'm not sure how well this will work. But, so far, this has cut down 350 lines of Makefile.am code.<br /><br />How does this work? First of all, I've written a script to generate the Makefile.am from the Makefile.am.m4 and put it in admin/generate-makefile.sh. All this script does is call M4, but I want to keep this logic in a single place because it has to be used from two call sites as described below.<br /><br />Then, I've added an autogen.sh script to the top-level directory that generates Makefile.am (using the previous script) and calls autoreconf -is. I'm against autogen.sh scripts that pretend to be smart instead of just calling autoreconf, but in this case I see no other way around it.<br /><br />At last, I've modified Makefile.am to add an extra rule to generate itself based on the M4 version. This, of course, also uses generate-makefile.sh.<br /><br />We'll see how this scales, but I'm so far happy with the results.<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-5706562320936440325?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Extending sudo credentials

Julio Merino — Mon, 16 Nov 2009 15:48:06 +0000

If you use sudo for, e.g. pkgsrc's just-in-time su, you may have often bitten by the problem that some compilations are slow and the build process stops right in the middle to ask you for a root password. If you go away while the system compiles, you'll be frustrated when you come back, as the process may still well be at the very beginning.<br /><br />This happens because, unless disabled by the system administrator, your sudo credentials last for 5 minutes. If you hadn't used sudo for those 5 minutes, it will ask you for your password again. A simple workaround for the problem is to automatically renew your credentials, say, every 2 minutes. You can do this by running the following command (from the same console you are using later on!) right before starting a pkgsrc build:<pre>$ ( while :; do sudo -v; sleep 120; done ) &</pre><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-190748834219167977?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Best config setting ever

Julio Merino — Mon, 16 Nov 2009 15:48:06 +0000

<pre>echo 'set editing-mode vi' >>~/.inputrc</pre>This will enable vi-editing mode for all commands that use the GNU readline library (e.g. bash, python, bc, etc.), not only the shell. For the shell only (including non-bash shells), add 'set -o vi' to your shrc file.<br /><br />I don't know why I didn't do this before given that I'm a pretty hard vi user. Still, for some reason, I kept using emacs-like key bindings for command-line editing. Not any more! However, be careful: if you are used to vim's visual editing mode, you'll keep hitting 'v' in the command line and getting super annoyed.<br /><br />Enjoy!<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-1338186479268297089?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): 1000 revisions for ATF

Julio Merino — Mon, 16 Nov 2009 15:48:06 +0000

Mmm! Revision <a href="http://mtn-host.prjek.net/viewmtn/atf/revision/info/7ca234b9aceabcfe9a8a1340baa07d6fdc9e3d33">7ca234b9aceabcfe9a8a1340baa07d6fdc9e3d33</a>, committed about an hour ago, marks the 1000th revision in the ATF repository. Thanks for staying with me if you are following the project :)<div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-681908603179050098?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Books by Joel Spolsky

Julio Merino — Mon, 16 Nov 2009 15:48:06 +0000

I just finished reading the third book in a row from Joel Spolsky, titled Joel on Software. Before this one, I read More Joel on Software and The Best Software Writing 1, all in a bit over a month. Note: I hadn't read any book cover-to-cover for a loooong while. Very interesting and entertaining books; highly recommended.<div><br /></div><div>Oh, and his writing style is really enjoyable. We, crappy blog writers, can learn a lot from him!</div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-9020484380911179497?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): Rearchitecting ATF

Julio Merino — Mon, 16 Nov 2009 15:48:06 +0000

During the last few weeks, I've been doing some ATF coding and, well... I'm not happy. At all. I keep implementing features but I feel, more and more, that ATF is growing out of control and that it is way too sluggish. It oughtn't be so slow. About 6 minutes to run the whole test suite in a Mac G3 I just got? HA! I bet I can do much, much, much better than that. Come on, we are targeting NetBSD, so we should support all those niche platforms rather well, and speed matters.<div><br /></div><div>The thing is, the current code base grew out of a prototype that didn't have that much of a design. Well, it had a design but, in my opinion, it has turned to be a bad design. I couldn't imagine that we could hit the bottlenecks (speed) and user-interface issues (for example, the huge difficulties that involve debugging a failing test case) that we are hitting. So...<div><br /></div><div><i>IT IS TIME FOR A CHANGE!!!</i></div><div><br /></div><div>I'm currently working on a written specification of what ATF will look like, hopefully, in the not-so-distant future. It will take a while to get there, but with enough effort, we soon will. And life will be better. And no, I'm not talking about a from-scratch rewrite; that'd only hurt the project. I plan to take incremental and safe steps, keeping the code base running all the time, but I will do a major face-lift of everything. (I wish I could say "we" instead of "I" here. But not there yet.)</div><div><br /></div><div>Why am I writing a specification, you ask? Well, because that forces me (or ANY developer) to think how I want the thing to look like and to decide, exactly, on what the design will be, which technologies will be used, which languages will be involved and in what components, etc. And no, I'm not talking of a class model design; I'm just talking about the main design of the whole picture, which is quite hard by itself. Plus having a spec will allow me to show it to you before I start coding and you will say "oh, wonderful, this new design sucks so much that I'm not going to bother with the new version". Or maybe hell will freeze and you will think, "mmm, this looks interesting, maybe it will solve these issues I'm having as regards speed, ease of debugging and ease of use".</div><div><br /></div><div>Anyway, I hope to have a draft "soon" and to hear any of the two possible comments as a result!</div><div><br /></div><div><b>Edit (July 29th)</b>: Alright, I have uploaded an <a href="http://www.netbsd.org/~jmmv/rearchitecting-atf/spec.html">extremely preliminary copy of the specification</a> just so that you can see where my current ideas are headed. Expect many more changes to this document, so don't pay too much attention to the tiny details (most of which aren't there anyway yet).</div></div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-4743147252783979430?l=blog.julipedia.org' alt='' /></div></content>

The Julipedia (Blog): The mess of ATF's code

Julio Merino — Mon, 16 Nov 2009 15:48:06 +0000

Yes. ATF's code is a "bit" messy, to put it bluntly. I'm quite happy with some of the newest bits but there are some huge parts in it that stink. The main reason for this is that the "ugly" parts were the ones that were written first, and they were basically a prototype; we didn't know all the requirements for the code at that point... and we still don't know them, but we know we can do much better. Even though I'm writing in plural... I'm afraid we = I at the moment :-P<div><br /></div><div>So, is it time for the big-rewrite-from-scratch? NO! Joel Spolsky wrote about why <a href="http://www.joelonsoftware.com/articles/fog0000000348.html">this is a bad idea</a> and I have to agree with him. Yeah, I'm basically the only developer of the code so everything is in my head, and I'd do a rewrite with a fresh mind, but... I'd lose tons of work and, specially, I'd lose tons of code that deals with tricky corner-cases that are hard to remember.</div><div><br /></div><div>Sure, I want to clean things up but they'll happen incrementally. And preferably concurrently with feature additions. These two things could definitely happen at the same time if only I had infinite spare time...</div><div><br /></div><div>Anyway, the major point of this post is to describe what I don't like about the current code base and how I'd like to see it changing:</div><div><ul><li>A completely revamped C++ API for test cases. The current one sucks. It is not consistent with the C API. It lacks important functionality. It uses exceptions for test-case status reporting (yuck!). And it's ugly.</li><li>Clear separation of "internal/helper" APIs from the test APIs. You'll agree that the "fs" module, which provides path abstraction and other file system management routines, is something that cannot be part of ATF's API. ATF is about testing. Period. Either that fs module should be in a separate library or should be completely hidden from the public. Otherwise, it'll suffer from abuse and, what scares me, will have to become part of ATF's API. And likewise, most &mdash; really &mdash; most of the modules in the current code are internal.</li><li>Less dependencies from the C++ API to the C API. Most of the current C++ modules are wrappers of their corresponding C counterparts. This is nice for code reuse but makes the code extremely fragile. In C++, things like RAII can provide really robust code with minimum effort, but intermixing such C++ code with C makes things ugly really quickly. I'd like to find a way to keep the two libraries separate from each other (and thus keep the C++ binding "pure"), but at the same time I don't want to duplicate code... an interesting problem.</li><li>Split the tarball into smaller pieces. People writing test cases for C applications don't want to pull in a huge package that depends on C++ and whatnot. And ATF is huge. It takes forever to compile. And this is a serious issue for broad adoption. Note: whether the tools are written in C++ or not is a separate issue, because these are not a dependency for anything!</li><li>The shell binding is slow. Really slow compared to the other ones. Optimizations would be nice, but those do not address the root of the problem: it's costly to query information from shell-based tests at run time. I.e. it takes a long time to get the full list of test cases available in a test suite because you have to run every single test program with the -l flag. Keeping a separate file with test-case metadata alongside the binary could resolve this and allow more flexibility at run time.</li><li>And some other things.</li></ul><div>Those are the major things I'd like to see addressed soon, but they involve tons of work. Of course, I'd like to be able to work on some features expected by other developers: easier debugging, DOCUMENTATION!...</div><div><br /></div><div>So, helpers welcome :-)</div></div><div class="blogger-post-footer"><img width='1' height='1' src='https://blogger.googleusercontent.com/tracker/17885055-8981481737885658422?l=blog.julipedia.org' alt='' /></div></content>