Adam Leventhal's blog
Search
Close this search box.

Month: October 2005

Fall of Code?

Actually, Fall, Winter and Spring of code. Sun just announced the Solaris 10 Univesity Challenge Content. That’s right, it’s a challenge and a contest and with three times the staying power of single season of code. Apparently in a modern day treaty of Brest-Litovsk, Google ceded the rest of the year to Sun. Perhaps this was the real beef of the recent Sun/Google agreement.

This is actually pretty cool: be a college student, do something cool on OpenSolaris, take a shot at winning the grand prize of $5k and a sweet box (I imagine there might be prizes for other interesting entries). There are some ideas projects on the contest/challenge/seasonal coding page ranging from good (MythTV), to mundane (support for inkjet printers from Epson), to confusing (internet gaming — I thought online gamling was its own reward), to inane (“A joystick driver – for gaming”, oh for gaming? I’ve been using it for system administration). Here’s my list off the top of my head — if you want more ideas, feel free to drop me a line.

Work on an existing open source project

  • pearpc runs ppc applications on x86. I started working on porting it over and was able to boot Darwin.
  • valgrind is very cool (I’ve only just seen it). It would be great to port it or to use pieces like KCacheGrind and plug in DTrace as a backend.
  • Port over your favorite application, or get it running in some emulation environment of your own devising.
  • Make something go faster. MySQL, Gnome, mozilla, some random system call, whatever; there’s a lot of inefficiency out there.

Write something new (using cools stuff in Solaris)

  • I’d love to see more dynamic languages with native DTrace support. We’ve already got support for Java, php, Ruby, and Perl in some form; make it better or add support for some other language you know and love (TCL, python, scheme, LISP, ML, etc.).
  • Build another kind of analysis tool on top of DTrace. We’re working on a Java binding which is going to make this easier.
  • Write a device driver for your favorite crazy device (which I assume is your new iPod nano or something; you’re such a hipster Apple fanboy).
  • Build a tool to simulate a distributed environment on Zones and use DTrace to monitor the communication. WARNING: your distributed systems professor will be your new best friend.

That’s what I’d do, but if you have a better idea, go do that instead.

Technorati tags:

OpenSolaris and svk

Today at EuroOSCON, I attended a introductory talk on svn by Chia-liang Kao. I was hopeful that svk might address some of the issues that I thought would prevent us from adopting Subversion for OpenSolaris. In particular, Subversion requires a centralized repository whereas svk, which is built on top of Subversion, provides the distributed revision control system that we’d need. After the talk, my overall impression was that svk seemed to lack a certain polish, but after struggling to phrase that in less subjective terms, I’m coming around a bit.

I got a little nervous when the first example of svk’s use was for keeping the contents of /etc under revision control. The big problem that svk solved was having random directories (.svc, SCCS, whatever) in, for example, /etc/skel. Talk about trivia (almost as relevant as a demo implementing a weblog in Ruby on Rails). I guess it’s nice that svk solves a problem for that particularly esoteric scenario, but wasn’t there some mention that this might be used to, you know, hold onto source code? Was that actually the design center for svk?

Fortunately we did get to the details of using svk for a source code repository. I think this is just my bias coming from teamware, but some of the mechanisms seem a bit strange. In particular, you do svk mirror to make one kind of copy of the main repository (the kind that’s a “local repository”), and svk checkout to make a different kind of copy (the kind that’s the “working area”). In other words, you have a tree structure, but the branches and leaves are different entities entirely and editing can only be done on the leaves. I guess that’s not such a huge pain, but I think this reflects the origins: taking a local revision control system and making it distributed. Consequentially, there’s a bunch of stuff left over from Subversion (like branches) that seem rather redundant in a distributed revision control system (don’t take branch, make another local repository, right?); it’s not that these actually hurt anything, it just means that there’s a bunch of complexity for what’s essentially ancillary functionality.

Another not-a-big-deal-that-rubs-me-the-wrong-way is that svk is a pile of perl modules (of course, there’s probably a specific perlism for that; “epocalyptus” or something I’m sure). I suppose we’ll only have to wrestle that bear to the ground once, and stuff it in a tar ball (yes, Allan, or a package). To assuage my nervousness, I’d at least like to be confident that we could maintain this ourselves, but I don’t think we have the collective perl expertise (except for the aforementioned Alan) to be able to fix a bug or add a feature we desperately need.

I want this thing to work, because svk seems to be the best option currently available, but I admit that I was a bit disappointed. If we were going to use this for OpenSolaris, we’d certainly need to distribute it in a self-contained form, and really take it through the paces to make sure it could do all the things we need in the strangest edge cases. As I mentioned, we currently use teamware which is an old Sun product that’s been in constant use despite being end-of-lifed several years ago. While I like it’s overall design, there’s a bunch of work that would need to happen for it to be suitable for OpenSolaris. In particular, it currently requires a shared NFS filesystem, and we require you to be on the host machine when committing a change. Adding networked access capabilities to it would probably be a nightmare. It also relies on SCCS (an ancient version control system) for managing individual files; not a problem per se, but a little crufty. Teamware is great and has withstood the test of time, but svk is probably closer to satisfying our requirements.

Update: But there are quite a few other options I hadn’t looked into. Svk no longer looks like a front runner. If there are other systems you think are worth considering, let me know.

I’ll play with svk some more and try to psych myself up for this brave new world. I’d appreciate any testimonials or feedback, and, of course, please correct all the factual errors I’m sure I committed.

Technorati tags:

OpenSolaris and Subversion

I just attended Brian W. Fitzpatrick‘s talk on Subversion at EuroOSCON. Brian did a great job and Subversion looks like a really complete replacement for cvs — the stated goal of the project. What I was particularly interested in was the feasibility of using Subversion as the revision control system for OpenSolaris; according to the road map we still have a few months to figure it out, but, as my grandmother always said while working away at her mechanical Turing machine, time flies when you’re debating the merits of various revision control systems.

While Subversion seems like some polished software, I don’t think it’s a solution — or at least a complete solution — to the problem we have with OpenSolaris. In particular, it’s not a distributed revision control system meaning that there’s one master repository that manages everything including branches and sub-branches. This means that if you have a development team at the distal point on the globe from the main repository (we pretty much do), all that team’s work has to traverse the globe. Now the Subversion folks have ensured that the over the wire protocol is lean, but that doesn’t really address the core of the problem — the concern isn’t potentially slow communication, it’s that it happens at all. Let’s say a bunch of folks — inside or outside of Sun — start working on a project; under Subversion there’s a single point of failure — if the one server in Menlo Park goes down (or the connection to it does down), the project can’t accept any more integrations. I’m also not clear if branches can have their own policies for integrations. There are a couple other issues we’d need to solve (e.g. comments are made per-integration rather than per-file), but this is by far the biggest.

Brian recommeded a talk on svk later this week; svk is a distributed revision control and source management system that’s built on Subversion. I hope svk solves the problems OpenSolaris would have with Subversion, but it would be great if Subversion could eventually migrate to a distributed model. I’d also like to attend this BoF on version control systems, but I’ll be busy at the OpenSolaris User Group meeting — where I’m sure you’ll be as well.

Technorati tags:

At Euro OSCON

This week I’ll be in Amsterdam attending and speaking at Euro OSCON. I’ll be running a session on DTrace on Thursday at 1:30. For the uninitiated, DTrace is the systemic observability framework in OpenSolaris that allows for tracing of the kernel and applications on production systems. At OSCON in Portland, Bryan and Wez extended DTrace into php (it already worked on C/C++ and Java); since then, the DTrace community has extended DTrace into Ruby and Perl — we’ll see if we can come up with something new this week.

I’ll also be tagging in on the OpenSolaris Users’ Group Meeting and the OpenSolaris Technologies BoF along with Casper, Alan, Patrick, Simon, and others.

I’ve had a bit of a chance to explore Amsterdam a bit and it’s spectacular. My California-thinned blood won’t let me forgo a comment on the rather cold weather, but my Northeastern upbringing won’t let me complain too loudly.

Technorati tags:

Too much pid provider

Perhaps it’s a bit Machiavellian, but I just love code that in some way tricks another piece of code. For example, in college I wrote some code that trolled through the address space of my favorite game to afford me certain advantages. Most recently, I’ve been working on some code that tricks other code into believing a complete fiction[1] about what operating system it’s executing on. While working on that, I discovered an interesting problem with the pid provider — code that’s all about deception and sleight of hand. Before you read further, be warned: I’ve already written two completely numbing accounts of the details of the pid provider here and here, and this is going to follow much in that pattern. If you skip this one for fear of being bored to death[2], I won’t be offended.

The problem arose because the traced process tried to execute an x86 instruction like this: 

call    *0x10(%gs)

This instruction is supposed to perform a call to the address loaded from 0x10 bytes beyond the base of the segment described by the %gs selector. The neat thing about the pid provider (in case you’ve skipped those other posts) is that most instructions are executed natively, but some — and call is one of them — have to be emulated in the kernel. This instruction’s somewhat unusual behavior needed to be emulated precisely; the pid provider, however, didn’t know from selector prefixes and blithely tried to load from the absolute virtual address 0x10. Whoops.

To correct this, I needed to add some additional logic to parse the instruction and then augment the emulation code to know how to deal with these selectors. The first part was trivial, but the second half involved some digging into the x86 architecture manual. There are two kinds of descriptor tables, the LDT (local) and GDT (global). The value of %gs, in this case, tells us which table to look in, the index into that table, and the permissions associated with that selector.

Below is the code I added to usr/src/uts/intel/dtrace/fasttrap_isa.c to handle this case. You can find the context here. 

1145                         if (tp->ftt_code == 1) {
1146
1147                                 /*
1148                                  * If there's a segment prefix for this
1149                                  * instruction, first grab the appropriate
1150                                  * segment selector, then pull the base value
1151                                  * out of the appropriate descriptor table
1152                                  * and add it to the computed address.
1153                                  */
1154                                 if (tp->ftt_segment != FASTTRAP_SEG_NONE) {
1155                                         uint16_t sel, ndx;
1156                                         user_desc_t *desc;
1157
1158                                         switch (tp->ftt_segment) {
1159                                         case FASTTRAP_SEG_CS:
1160                                                 sel = rp->r_cs;
1161                                                 break;
1162                                         case FASTTRAP_SEG_DS:
1163                                                 sel = rp->r_ds;
1164                                                 break;
1165                                         case FASTTRAP_SEG_ES:
1166                                                 sel = rp->r_es;
1167                                                 break;
1168                                         case FASTTRAP_SEG_FS:
1169                                                 sel = rp->r_fs;
1170                                                 break;
1171                                         case FASTTRAP_SEG_GS:
1172                                                 sel = rp->r_gs;
1173                                                 break;
1174                                         case FASTTRAP_SEG_SS:
1175                                                 sel = rp->r_ss;
1176                                                 break;
1177                                         }
1178
1179                                         /*
1180                                          * Make sure the given segment register
1181                                          * specifies a user priority selector
1182                                          * rather than a kernel selector.
1183                                          */
1184                                         if (!SELISUPL(sel)) {
1185                                                 fasttrap_sigsegv(p, curthread,
1186                                                     addr);
1187                                                 new_pc = pc;
1188                                                 break;
1189                                         }
1190
1191                                         ndx = SELTOIDX(sel);
1192
1193                                         if (SELISLDT(sel)) {
1194                                                 if (ndx > p->p_ldtlimit) {
1195                                                         fasttrap_sigsegv(p,
1196                                                             curthread, addr);
1197                                                         new_pc = pc;
1198                                                         break;
1199                                                 }
1200
1201                                                 desc = p->p_ldt + ndx;
1202
1203                                         } else {
1204                                                 if (ndx >= NGDT) {
1205                                                         fasttrap_sigsegv(p,
1206                                                             curthread, addr);
1207                                                         new_pc = pc;
1208                                                         break;
1209                                                 }
1210
1211                                                 desc = cpu_get_gdt() + ndx;
1212                                         }
1213
1214                                         addr += USEGD_GETBASE(desc);
1215                                 }

The thing I learned by writing this is how to find the base address for those segment selectors which has been something I’ve been meaning to figure out. We (and most other operating systems) get to the thread pointer through a segment selector, so when debugging in mdb(1) I’ve often wondered how to perform the mapping from the value of %gs to the thread pointer that I care about. I haven’t put that code back yet, so feel free to point out any problems you see. Anyway, if you made it here, congratulations and thanks.

[1]Such is my love of the elaborate ruse that I once took months setting up a friend of mine for a very minor gag. Lucas and I were playing scrabble and he was disappointed to hear that the putative word “fearslut” wasn’t good. Later I conspired with a friend at his company to have a third party send mail to an etymology mailing list claiming that he had found the word “fearslut” in an old manuscript of an obscure Shakespear play. Three months later Lucas triumphantly announced to me that, lo and behold, “fearslut” was a word. I think I passed out I was laughing so hard.

[2]My parents are fond of recounting my response when they asked what I was doing in my operating systems class during college: “If I told you, you wouldn’t understand, and if I explained it, you’d be bored.”

Recent Posts

Austin API Summit Wrap-up

April 17, 2024

Rust and JSON Schema: odd couple or perfect strangers

January 22, 2024

Oxide and Friends Season 4

January 13, 2024

DTrace probes in Rust

December 29, 2023

From Prometheus to Sisyphus

February 12, 2017

DTrace at Home

December 18, 2016

Archives

Archives
Adam Leventhal's blog