Surprisingly, static typing is not necessary to get this selective, parameter-by-parameter behavior! To see this, consider an implementation approach where you always thunk-ize the arguments to method invocations, just in case. Then, when a method invocation actually runs, it does the following:

1. Evaluate the receiver.
2. Determine exactly which method that respond.
3. Look at that method's parameters, and evaluate the parameters that are marked "by value".
4. Invoke the method, passing values for the by-value parameters and thunks or the by-name parameters.

To see how it works, here is how you would modify lambda calculus. Lambda calculus has functions instead of methods, but the idea works out the same. Recall the standard evaluation rules for lambda calculus:

(\x.e) v  -->  e[x\v]   
e1 e2  -->  e1 e2'      (assuming e2 --> e2')
e v  -->  e' v          (assuming e --> e')

On top of these, you can add a "by name" lambda expression, \BN x.e, which has the following evaluation rule:

(\BN x.e1) e2  -->  e1[x\e2]

That is all. Note two differences between this rule and the ones for normal, by-value lambdas. First, you can pass an arbitrary expression into the lambda's body, not just a value -- that's by-name evaluation. Second, there is no recurrence for evaluating the argument. Once it is clear that the callee is a by-name lambda, it can go ahead and be invoked.

The net result is that an application (e1 e2) is evaluated as follows. First, e1 is fully evaluated into either a lambda or a by-name lambda (if it is something else, computation stops). Then, one of two things happens. If e1 is a normal by-value lambda, then e2 is evaluated and then the result is passed into the lambda. If e1 is a by-name lambda, then e2 is immediately passed into the body of that lambda.

The decision is made at run time, and so the same expression might even use a different evaluation order each time it runs. It is late binding of evaluation order.

After thinking about all of this, I am left with two questions:

1. Is there a fast implementation? Thunking every single parameter looks expensive, but there might be better implementations.
2. Is there any language feature where you need static typing other than for checking and for speed? I used to say parameter-specific execution order was an example, but now I see that it is not. What is left?

The study claims that iPods interfere with pace makers. Thaker, a high school student, worked with professional medical researchers to perform a study on 83 people. This is a very interesting topic, and one I would like to read more about the details. Here are some key, easy questions that lept to my mind, questions that the original paper could answer quickly:

1. What kind of pace makers did he study? Pace markers are not government issue and completely identical.
2. Did he test pace makers standing alone, to try and find a mechanism for what he observed?
3. How do the observed anomalies compare against the pace makers' normal inaccuracies? Saying that x% have anomalous behavior is extremely interesting, but I want to know more about what the anomalies are.
4. Did he verify, or at least argue, that the interference was not affecting the diagnostic machines? I expect they ruled this out, but would like to be reassured.

The study writeup itself does not come up on Google. More disturbing, though, is that the news articles I read neither link to the article, nor present an excuse for not doing so. Hyperlinked media is about linking. If anyone knows a good news outlet that is good about this, I would like to know.

Also disturbing is the lack of fact checking going on. I did not run into a single article that linked even to the conference where the paper was presented. I wonder if they even bothered to check, because in fact the conference web page does not list Thaker as a presenter! When they say that Thaker "presented" the article at the conference, they must mean that he had a poster, or that he walked around and chatted with the researchers. If he gave a formal presentation, it is not listed on the schedule.

A good news outlet on the web ought to at least link to the materials they are talking about. If they discuss a study, link to it. If they discuss a bill before Congress, link to the bill. If they discuss a novel, link to the Amazon page for the novel. It is hard to find even this level of support from news outlets.

A really good news outlet could go further, and dig up relevant expert material. They could offer more than the pithy one-line quote that on-paper newspapers give you, and link to an entire expert testimony. For example, I would really like to know whether the experts in this area think the level of interference Thaker found is a surprise.

Kudos to Jay for a really cool study. I am left wishing, though, to find a news outlet that is willing to have external hyperlinks. Please email me if you have any pointers. ]]> April 6, 2007 http://www.lexspoon.org/blog/dell-linux.htmlI just noticed that on Dell's Idea Storm site, quite a lot of the proposals are about pre-installing Linux on their machines. I wonder if Dell will go for it?

I have long been interested in the idea of pre-installed Linux machines. Right now, if you want to use Linux, you face two learning curves: using Linux, and installing Linux. Installing is actually the larger of the two learning curves for many people. If you plan to simply use pre-configured stuff from Ubuntu, then time spend on installation details is mostly wasted. I use Debian, on which Ubuntu is based, and it has taken many tweaks to get my Thinkpad into a good state.

So should Dell do it? My thoughts about the good and bad for Dell:

• Good: Dell would occupy a mostly uncontested niche: laptops with normal Linux installations.
• Bad: Dell would add a constraint on the hardware they include, because it would be embarrassing to ship a Linux laptop where some of the hardware does not have drivers. For hardware that Just Works in Linux, they will have a low burden. For other hardware, they will have to either disable it, or convince the manufacturer to develop a driver, or get the Windows driver to work in Linux, or develop their own driver. This constraint can play out many ways, but the net effect is an extra burden on Dell.
• Bad: The Linux side would likely be unpolished. Dell will need some careful spin control to keep their brand name from being dragged down by the flaky parts. Perhaps the line they can take is: the Linux side is for those who want to take off the hood and get their hands into the machine. As an alternative: the Linux side is for those who want to experiment with the raw future of computing.
• Good: "Open source" has a positive connotation much like environmental protection or poverty relief. Microsoft got trashed for giving away their web browser for free; Netscape and IBM have gotten kudos for equally predatorially [1] giving away software, but also making that software open source. Dell would rather be the IBM or Netscape in this picture, not the Microsoft.

The Unix system has always had its user-space look and feel defined by the Init process (typically in the program file /etc/init) which spawns and defines the configuration of the normal expected Unix environment. The development described here proposes to replace this set of user-level processes with a Lisp system. This Lisp-based environment will then be used as a new process to configure and define an environment similar to the Lisp Machine's in the user space provided by the Linux kernel.

I laud the goal, but I question the path. Yes, there are real advantages to intercepting init. You get to reuse the Linux kernel, which should be a perfectly fine kernel for something like a Lisp Machine, and you can reuse the Lisp Machine design for your reimplementation.

However, the next step requires an ugly choice. If you try to stay true to the Lisp Machine design, then you get the advantage of using a tried and true design, but you have a lot of reimplementation to do, and you will end up with an old system that predated the rise of the Internet.

Better, you can try to design an entirely new set of low-level utilities. This has the downside of being a from-scratch design, and of require quite a lot of work before you have anything you can really use. However, this approach is doable. It looks a lot like designing a new Linux distribution.

My thoughts have always been, though, to implement hacker's OS by intercepting at a higher level: the user shell. I mean "shell" in a general sense, as that part of the software that interacts with the user, and thus I would include web browsers, window managers, and so on as shells. If you write a shell, then you can start modestly and then progressively write and incorporate more tools in a Lisp Machine style. This approach is just as powerful in the end, and it avoids a lot of costly mis-prioritized design, and it has the advantage that you get to play immediately.

So, I think of a hacker's OS as starting with something like Emacs, or Squeak plus Command Shell.

Anyway, I wish Egdorf the best of luck. If he does no more than make a Linux distribution with Lisp at the core, that would be a marvelous step forward for personal operating systems. ]]> March 13, 2007 http://www.lexspoon.org/blog/perform-copyright.htmlI wrote before that copyright is poorly designed now that digital processors are ubiquitous. Our current copyright law makes sense for paper books because they are hard for individuals to copy. They do not work well for digital material, however, because digital material is extremely easy to copy. There is only a slim hope of applying traditional copyright to digital works, and that is to heavily restrict the use of all kinds of digital devices. Heavy restriction does not lend itself to a pleasant society.

The PERFORM Act, recently introduced to the U.S. Senate is an example of propping up existing copyright approaches. The PERFORM Act addresses broadcast of music by digital radio stations such as 99x.com. Currently, such stations are free to broadcast in open formats. The bill would require that they use awkward formats that attempt to prevent copying.

The prevention of copying is impossible, of course, so long as playback is possible, but the act would require stations to make "reasonable" efforts at the impossible. The efforts are onerous for the stations, and lead to limited and cumbersome software on the machines of end users.

We should strive to design copyright law that makes sense with current technology. Personal computers are here, and they are great and copying digital data. These are positive changes! We should change the law, not try to turn back technology.

More information is available via the Electronic Frontier Foundation. ]]> February 1, 2007 http://www.lexspoon.org/blog/real-name-wikipedia.htmlponders accountability on Wikipedia, and suggests that it use real names:

The first, obvious step is accountability. For starters, a real name. At some point, we are going to need more: a domain, and address, and a footprint on the Web so we can see clearly who the police are and what they are doing.

Accountability is important, but Bernstein's list of mechanisms needs to be reconsidered. The thing you really need for accountability is a good, unforgeable record of what an account holder has done in the past. I bet Wikipedia already has this. Probably, every edit already is logged along with who performed it. If so, then Wikipedia already has accountability, and "anonymous" is a misnomer. What could be less anonymous than logging every single activity along with the name of who did it?

Looking for "real" names is a dangerous distraction. The other information in Mark's list is personal and frequently not helpful. There are exceptions; for example, you might use your real name to establish expertise in an area. There are even larger grounds for prejudice, though. Imagine you find out the human behind a Wikipedia account and you learn that they are, say,.... a religious fundamentalist? ....twelve years old? ....a political activist?

Additionally, keep in mind that some people live in uncomfortable situations. Not everyone gets to sit in a university and pompously declare to the world whatever is on their mind today. Some people need to profess the local faiths, and for them a pseudonym is the only way they can speak freely. If real names become mandatory, then many good contributors will simply be unable to contribute.

Overall, pseudonyms are not only acceptable on Wikipedia, but productive. I would go further and say they are a general feature of online life. If that sounds strange at first, then please consider a more familiar concept: separating private life from business. Is it really so much of a stretch to separate even more lives? ]]> January 30, 2007 http://www.lexspoon.org/blog/guzdial-on-why.htmlMark Guzdial, one of my graduate school advisors, just posted on why computing education is important to him.

No, we're not about curing small children. But we are creating the infrastructure for a lot of important work. Literacy is powerful. The people who invented writing did not cure disease, but by inventing writing, they made all of science possible, not just medicine. Perhaps the medical researcher who first understands how autism works will come to that understanding or will communicate it by creating a computational model of autism. And maybe that medical researcher is in our introductory computing classes today.

His answer fits well with mine. Computing is a new tool for thought, all kinds of thought. It helps scientific research, business, education, art---anywhere a mind is active, computers have found a way to help. Computing will be as fundamental as writing, so it is worth teaching the craft as we know it, and it is worth trying to improve that craft. ]]> January 28, 2007 http://www.lexspoon.org/blog/coyotedigitalcopy.htmla post today about digital copyright. Computer scientists continue to decline to propose any meaningful reform of digital copyright. As a result, the extension of obsolete copyright law plays on out, an the general public is noticing more as time goes on. We see our computing and playback systems--a distinction increasingly blurred--fall under limitation just because it might be used to violate copyright. We have an increasingly difficult time using content that we have paid for. ]]> January 2, 2007 http://www.lexspoon.org/blog/digital-copyright.htmlCopyright is important, but the current approach to it is out of date now that computers make copying so cheap. The way copyright is designed impacts any business involving digital material, including software, books, music, and video.

In the form we currently know it, copyright is only a few hundred years old. Currently, if you write a book or put a string of notes in a row, then you automatically have the sole copyright -- the right to copy -- for that work. You can then license permission to others to make their own copies, typically charging them a fee for each copy. The better your work, the more you can charge. Thus, you get paid for producing good material.

To contrast, musicians of just a few hundred years ago had no protection against other people copying their material. Bach could not and did not get rich by selling copies! The main barrier to copying was the sheer difficulty of making a copy at all. Before the days of mass production, this barrier was significant.

There is no reason that we must continue with the current approach to copyright. Yes, existing businesses crucially rely on the current approach to copyright. However, this approach is based on technology of the past.

A particularly important change is that copying has become cheap and easy. Fifty years ago, making a copy of a book was an expensive operation. Now, you simply Google with "site:.ru" in your search field and you can find a copy of the book to download.

This change strikes deep at the design of copyright. When copying requires a major enterprise, violation for personal use is not worth the cost, and violation on a massive scale is easy to detect. Nowadays, however, copies of all popular works are freely available on the Internet, even though everyone posting such works is paying the risk of living outside the law.

If we continue with the current approach to copyright, then some outcomes are predictable. All means of content copying and distribution will get closer inspection. Since all computers are also copying devices, the devices themselves will come under legal oversight. People will not be able to own just any old computer any longer, and people who buy parts for hobby machines will be viewed just like those who today buy ingredients for bombs or for illegal drugs. Further, popular computer software will include more attempts at digital-rights management (DRM). For example, Windows Vista disables itself if it decides -- correctly or incorrectly -- that the user is no longer allowed access. Users do not precisely buy a copy of Windows Vista; they lease access to it.

As a result, insisting on our current approach to copyright will, perversely, undermine our current intuition about copies. Traditionally, people buy and own copies of a work, and they can do with those works as they please up to the point of making and distributing new copies. Now that copying is cheap, however, people will increasingly find that they do not own copies in the traditional sense. "Their" copies will disable themselves without the user requesting. "Their" copies will be uncopyable, even for purposes of making backups.

It is a major problem, and well worth careful consideration. Let me start by sketching just two ideas about redesigning copyright. I hope this discussion is picked up by computer scientists and legal theorists, so that we can design a system that will work well.

As one comment, note that it is not a terrible choice to simply abandon copyright. After all, the current approach works best in a world where copyable materials are generally mass produced, but that world is passing. As with any change to the rules, current businesses in the area will have to majorly change or go out of business. In this case, big-budget, mass-produced works like Microsoft Windows and The Matrix would no longer be profitable. However, there would still be plenty of software available and plenty of film. At the least, open source software and creative commons artwork would be available. Art would not die, but big-budget art mostly would.

The first idea is mainly meant as a starting point, an example to show that there are copyright designs that are simple and have good results. If we cannot come up with some good redesign of copyright, then we can abandon copyright entirely and do quite well.

A second idea is to change the focus from the right to copy to some other kind of right. We would need to choose an activity that is cheap for mass production but expensive for personal use, akin to the way copying itself used to be expensive for personal use. For example, perhaps we should control the right to publicly produce a work? In that case, The Matrix would have a business model but Microsoft Windows would not. Whereas people could own DVD's of The Matrix for free, movie theaters would have to pay to show it. The basic formula of traditional copyright would hold, and the system should work out equally well.

At any rate, the current design of copyright is flawed, and the problems are only going to grow as copying gets even cheaper. Computer scientists, who understand this material so well, should be at the forefront of efforts to redesign copyright to match current technology. If we find nothing to propose, then we should expect the trends to continue: full ownership of copies will decline, and surveillance and control will increase for our personal machines. ]]> December 31, 2006 http://www.lexspoon.org/blog/cupsconfig.htmlI am trying to set up my Linux laptop to print to my fiancée's Windows computer, and have stumbled deep into the configuration swamp that is the Common Unix Printing System, CUPS.

At this stage, I am stuck trying to find a "PPD" file, whatever that is, for my printer. I had a lot of trouble finding one at all, and finally I found a link to Lexmark's web site (which I did not find when browsing their site directly), but the downloaded link is a shell file that fails to extract itself.

There are many reasons this process is maddening. A big part is due to the client needing a printer-specific driver at all. This is some kind of mis-design in the SMB protocol, because a good network protocol should itself serve as the driver. I should be able to tell the client the server and printer name, and then the client should then be able to ask the server for whatever it needs. I should not need to tell the client stuff that the server already knows.

This alone would not be so terrible, if only CUPS itself were not so difficult to configure. There are many aspects to this difficulty, too, but high among them is the confusing web-based configuration tool. I am doing one of the simplest, most common of configurations: making a Windows printer visible on my Unix machine. This should be easy, but at least an hour later I am still failing.

During all the googling I have done this afternoon, I found an excellent article by Eric Raymond about my precise situation. Here is one of the central points:

The meta-problem here is that the configuration wizard does all the approved rituals (GUI with standardized clicky buttons, help popping up in a browser, etc. etc.) but doesn't have the central attribute these are supposed to achieve: discoverability. That is, the quality that every point in the interface has prompts and actions attached to it from which you can learn what to do next. Does your project have this quality?

This strikes me as a great mental trick for designing this kind of user interface. The CUPS team may well have great technical talent, but they could benefit greatly from having a UI designer who concentrates on issues like this.

For me, at this point, the next thing to try will be to put some other kind of server on the Windows machine--maybe LPD, or maybe this IPP thing CUPS keeps talking about. This step should not be necessary, but hopefully it will make things easy enough for CUPS that the rest goes smoothly.

I cannot escape the feeling, though, that this should all be easier. I also wonder, is there a better, more comprehensible printing system for Linux than CUPS? Emails welcome if you know of one! ]]> December 1, 2006 http://www.lexspoon.org/blog/batch-substitution.html

if "%OS%"=="Windows_NT" (
  rem ...stuff...
) else (
  set _SCALA_HOME=%SCALA_HOME%
  if "%_SCALA_HOME%"=="" goto error1
)

This code sets _SCALA_HOME from SCALA_HOME, and then tests whether _SCALA_HOME is non-empty. You would think that if SCALA_HOME were non-empty, then the second "if" statement would never fail.

You would be wrong. Before entering an "if" or "for" branch, Windows goes ahead and substitutes all variables. For example, if you suppose SCALA_HOME starts as "C:\scala" and _SCALA_HOME starts as "", then the above "else" clause would get substituted like this:

  set _SCALA_HOME=c:\scala
  if ""=="" goto error1

Notice that this second "if" will call error1 regardless of what _SCALA_HOME gets set to!

Keith figured out the following workaround:

) else (
  set _SCALA_HOME=%SCALA_HOME%
  rem The following line tests SCALA_HOME instead of _SCALA_HOME, because
  rem the above change to _SCALA_HOME is not visible within this block.
  if "%SCALA_HOME%"=="" goto error1
)

I wonder how many hours of productivity have been lost around the world due to this mystery? We have lost about 10 man-hours internally in the Scala group, and it looks like Keith lost another few himself. It is indisputable that Windows remains a viable operating system, a topic which cuts to the heart of efforts to make better programming languages. Leaving that topic aside, I still wonder: how much better could Windows be if its core scripting language had intuitive semantics?

Personally, I would love to take this idea further, and see a really nice shell language. It seems that Windows could get a productivity boost by simply adopting the decades-old Bourne shell that is used in Linux distributions. Linux users should not be complacent, though; Scheme Shell, for example, can boost shell-scripting productivity for those who adopt it. Looking further ahead, a good shell is a crucial part of what I think of as the hacker's OS of the future. A good shell is what makes Smalltalk and the Lisp Machine be more than just good languages, but good platforms for hacking. ]]> October 25, 2006 http://www.lexspoon.org/blog/guysteele06.htmlI enjoyed hearing Guy Steele talk this morning about his plans with Sun's new Fortress language. Fortress is a modern language for scientific computation. Steele plans to support the usual scientific-computation things efficiently, e.g. floating point arithmetic and matrix operations, while having a system that can support niceties of current languages such as objects and garbage collection.

Steele mentioned two interesting philosophical stances. First, Fortress puts things into a library instead of the compiler whenever possible. Second, the standard library itself is divided into components which can be individually updated. One audience member asked if this arrangement will result in multiple dialects. Steele says YES, Fortress will have multiple dialects. He considers this a lesser evil than having a rigid language incapable of adjusting to new ideas for improvement. Besides, Linux also has dialects via its different distributions; only a moderate number of distributions are important enough for most Linux users to ever learn.

There are a ton of cool individual elements of Fortress. To mention just one, the types include unproven properties such as "symmetric" and "associative". This way, users can define numeric types like rational numbers, and library routines can define optimizations that only take effect if the arguments have the necessary properties. Normally such knowledge is embedded in the compiler, giving the primitive types a major compilation advantage over user-defined types. Will this work out? Who knows, but it seems to be working so far!

Finally, Fortress has syntax. Optional terminating semicolons, a favorite flame war within LAMP, is only the beginning. Fortress allows arbitrary Unicode, like Java, but unlike Java it really embraces it. The library routine for summation, for example, is the Greek sigma. If you want to talk about the real numbers, you write a black-board R instead of the regular R. But why stop there? On top of all of this, Fortress has significant white space, and it distinguishes italics letters from boldface letters.

Overall, Fortress is a very interesting language. It is ambitious in many ways, but if anyone can make it all work out, it is Guy Steele. ]]> October 22, 2006 http://www.lexspoon.org/blog/lcsd06.htmlLCSD is a workshop of OOPSLA on libraries. Libraries have long been a core concept for practical software development, and they are especially important as more software is built by teams distributed around the world. I enjoyed hearing what other people are thinking about libraries nowadays.

My talk on interface evolution generated several questions and responses. Many people seem to be familiar with this issue!

The keynote was by Sean Parent from Adobe. The theme of his talk is that the glue code is the code. He went on from there to talk about some experiments Adobe is making with a declarative language for gluing together GUI environments. It addresses the same problem as SuperGlue.

Several other talks I found interesting for various reasons. Lubomir Bourdev and Jaakko Jarvi talked about the tension between generic programming, size of compiled code, and efficiency. They discuss their solution in this design space, based on C++ templates. Cosmin Oancea and Stephen Watt talked about inter-library interfaces that have a lot of abstraction, even when your interface language is a lowest-common-denominator system like CORBA. The speaker plugged Scala's triple of abstraction mechanisms for having extensible components: type members, mixins, and GADT's.

Eric Van Wyk, Derek Bodin, and Paul Huntington talked about their system for having language extensions in libraries. This is exciting because the Scala group is exploring things like language extension at the language level. The system of Wyk, et al., is based on putting attributed grammar fragments in the libraries. If you import a library, then you get access to the syntax and semantics defined in those attribute grammar fragments. Interesting. To contrast, Scala's efforts use fixed Scala syntax, and the semantic changes are implemented in general Scala code, not in a separate attribute language. ]]> October 13, 2006 http://www.lexspoon.org/blog/interfevol.html In Java when you add a new method to an interface, you break all your clients.... Since changing interfaces breaks clients you should consider them as immutable once you've published them. --Erich Gamma

NoSuchMethodError --Java VM, all too frequently

The two quotes above are problems of interface evolution. The first one suggests a real problem in Java's interface approach. At least one master designer is reluctant to use them, and indeed Gamma has gone on the praise Eclipse's replacement of Java interfaces by descriptors written in XML. There is an implicit put down here: Java's interfaces are bad enough that master designers write their interfaces in a separate language.

The second quote shows up all too frequently on Scala Bazaars. People independently post compiled jar's onto a bazaar, and all too frequently we discover that different versions of compiled jars are incompatible with each other. Their interfaces have shifted and are no longer compatible.

Both of these problems are caused by interface evolution. Either you get stuck with a bad interface, or you improve the interface and break compatibility.

There is a lot that can be done to ease the pain of interface evolution. One tool in the tool box is to extend the idea of deprecation into transition checking. With this approach, you step into interface evolution carefully, and you have a checker to help you decide when to take the plunge and make an API-breaking change. I will be talking about this approach at the end of the month at the Workshop on Library-Centric Software Development:

Spoon, S. Alexander. Anti-Deprecation: Towards Complete Static Checking for API Evolution. Workshop on Library-Centric Software Development (LCSD) 2006. (extended version with proofs)

Abstract: API evolution is the process of migrating an inter-library interface from one version to another. Such a migration requires checking that all libraries which interact through the interface be updated. Libraries can be updated one by one if there is a transition period during which both updated and non-updated libraries can communicate through some transitional version of the interface. Static type checking can verify that all libraries have been updated, and thus that a transition period may end and the interface be moved forward safely. Anti-deprecation is a novel type-checking feature that allows static checking for more interface evolutions periods. Anti-deprecation, along with the more familiar deprecation, is formally studied as an extension to Featherweight Java. This formal study unearths weaknesses in two widely used deprecation checkers.

In fact, it is far easier than ever to learn to program. When I started, my computers and calculators each had one or two programming languages, and as a child I had no opportunity to add a new one. I got BASIC, Pascal, or a calculator's custom language, and that was it. I cherished each scrap of example code I came across. My textbooks did not have program listings at all; I would have loved to have the scraps of BASIC that Brin talks about.

Nowadays, we have the Internet, and we have much better learning languages than the BASIC interpreters Brin talks about. The Internet gives access to gobs of tutorial information and example code. It also supplies great programming systems for newbies, like Squeak and Dr. Scheme, not to mention the various Logo's.

A better conclusion from Brin's essay is that teachers are using out of date text books. BASIC was flamed decades ago as being a poor language, and people who want to learn to program are drifting away from it. Whet he should conclude is that the curriculum pipeline is poor (they chose BASIC instead of Logo) and too long (there are better languages now).

But instead of this conclusion, Brin attacks researchers:

If this is a test, then Ben and I passed it, ingeniously. In contrast, Microsoft and Apple and all the big-time education-computerizing reformers of the MIT Media Lab are failing, miserably. For all of their high-flown education initiatives (like the "$100 laptop"), they seem bent on providing information consumption devices, not tools that teach creative thinking and technological mastery.

Researchers always risk being out of touch. However, Brin is attacking their strong point. Two of the three systems I mentioned above are spearheaded by people on this very same $100 laptop project: Alan Kay did Squeak, and Seymour Papert did Logo. On top of that, he explicitly mentions the MIT Media Lab, out of which came the great Lego Mindstorms. Brin is apparently unaware of what the researchers are actually doing.

The quoted comment about the $100 laptop points to where his core argument goes wrong. Yes, the $100 laptop is mainly an information sharing device, but that is because it is targeted at a general audience. Most computer users are not programmers!

And there we are. Yes, the proportion of computer users who are programmers is small nowadays. However, this is only because the increase in programmers is dwarfed by the increase in computer usage in general.

Everyone today is a computer user, and there are more programmers than ever. ]]> July 1, 2006 http://www.lexspoon.org/blog/net-neutrality.htmlNeutrality in general

I can believe that the layered nature of the Internet is one key to its high popularity and capability. It is tempting for network designers to try and make networks smart in various ways. TCP/IP, however, mostly restricts itself to moving packets around from address A to address B. The one feature it offers is retransmission to ensure reliable delivery, and that feature is optional.

The result is that there is a marketplace of network services to run on top of TCP/IP. Anyone, anywhere in the world, can start their own Internet service. The good ones become Googles. The poor ones become Floater bridge networks. There are a lot of good ones.

Network neutrality proponents fear that this aspect of the Internet will disappear soon. They fear that a few companies in the U.S. could, through offering different prices to different customers (e.g., charging more if you appear to be wealthier), could cause competition to be stifled.

The last mile

To clarify matters, I should say from the outset that phone and cable lines—the industry targeted by the original proposal before Congress—typically have a local monopoly. Thus, their behavior in locales can and should be carefully regulated.

The question I wish to address, though, is for nationwide companies, companies like Verizon, AT&T, and Cingular. At such scales there is no strong reason to grant a monopoly. Any company can lay down a new cross-country network. There is a case for separating inter-city networking from local networking, and the standing policy in the U.S. for phone networks is precisely that.

Efforts in the past

Since the feared problem has never occurred on the Internet, we can only speculate about the future. One way to do this is to look at efforts in the past by companies to segment the Internet.

Prodigy, CompuServe, MSN, and AOL are examples that leap to my mind. They all had their own network, and they all gave customers only restricted access to the rest of the Internet. What became of these companies?

1. Prodigy and CompuServe are dead as far as I can tell. They never changed and their customers simply left.
2. MSN gave up network service providing and went into other business.
3. AOL stopped their segmentation efforts, and now gives full Internet access.

The available evidence is that when network providers try to isolate their customers, the customers leave.

Defining a solution

I am at a loss to even imagine a regulatory structure that would force better network neutrality than we already have. Proponents, on the other hand, seem to find it obvious.

I believe that proponents might be thinking of the Internet as looking like a star:

With such a topology, you can control the Internet by controlling the central node. However, the Internet is actually more like a spiderweb:

The net in Internet means network. The Internet looks like a big spiderweb. Who, in this web, is supposed to be regulated? And how? The very design of the network is to switch packets. Each router handles packets in its own way, and each router has easier access to some parts of the network than others. Since giving equal access is impossible, what exactly would the regulation require?

To rub it in, try considering some specific networks segments you may be familiar with. Most companies and universities, for example, have their own internal networks plus a router connecting them to the Internet. The internal network is a bona fide part of the Internet, but existing practice is to treat internal packets differently from packets coming in and out of the institution. That is to say, the majority of existing networks are only effectively network neutral, but technically they include discrimination against different packets. How would you tailor regulation so that the current beneficial behavior does not become outlawed?

For that matter, what about the wireless routers within individuals' homes? Why should they not have to provide network neutrality themselves?

Given the web-like and not star-like topology of the Internet, controlling its behavior is difficult at best.

More likely solutions

In the previous section I tried to imagine the best possible regulation, but realistically we cannot assume that Congress will find the best possible regulation. Let us take a moment to look at previous government attempts to regulate networks, to gain a more likely picture of what they would accomplish.

Consider first the Communication Decency Act (CDA). The CDA attempted to control the content exchanged on the Internet, but there was no practical way for most service providers to implement the requirements. Thus, the CDA, if it had stayed in effect, would have forced most services like livejournal or blogspot to disappear to go out of business. The Supreme Court struck down the CDA, but the CDA gives us an idea of the kind of stifling law that Congress considers par. They do not seem to understand the Internet.

Consider the Digital Millennium Copyright Act (DMCA). The DMCA simultaneously allowed backups of DVD's but disallowed copying DVD's. Which is it? These two parts of the DMCA contradict each other, but that is acceptable to Congress when it comes to technical matters. Congress does not appear to understand software.

Consider the Federal Communications Commission (FCC). The FCC provides important administration of limited wireless bandwidth. However, in doing so, they must inevitably make decisions about who may use that bandwidth. Over time, they have come to enact decency provisions analogous to the CDA. To see the effect, compare the bland, inoffensive content on broadcast television to the more highly varied content on cable television. Do we really want an FCC for the Internet?

Finally, the elephant in the room is the current telephone network. For various reasons, U.S. governments at all levels have applied a sophisticated web of regulations to the telephone network. Among many other results, it is now cheaper to make a phone call over the Internet than via the dedicated, carefully coddled phone network. The effort to "save" the telephone network through regulation has resulted in a worse network than the much younger Internet. Instead of making the Internet more like the telephone network, perhaps we should embrace its key differences.

Can any readers point me to insightful network or software regulation that Congress has passed?

Related Reading

Those are the main things I want to say. There has been much discussion around the Internet.

I like Arnold Kling's take as an economist, and I sympathize with Jim Robertson's frequent snarks about an "FCC for the Internet".

On the flip side, there are many big names proposing regulation, but their arguments are troublingly weak. (Take this as a challenge, readers—tell me about arguments you find strong, and I will link to them.)

Lawrence Lessig provides one example. Lessig first equates open source to GNU-like licenses, a bizarre claim which means that Apache, FreeBSD, and Scala are not open source. Then, he claims that the GPL "regulates" software, even though "regulation" usually applies more to government mandates than to voluntary private agreements. I have never seen a starker example of doublespeak.

Tim Berners-Lee makes an eloquent and compelling argument about layered architectures--far better than my feeble attempt above--but he fails to address what exact kind of regulation he would like. Indeed, he himself writes:

To actually design legislation which allows creative interconnections between different service providers, but ensures neutrality of the Net as a whole may be a difficult task.

If you want flame more than light, go visit savetheinternet.net.

Finally, many, many links can be found from the Wikipedia article. ]]> June 28, 2006 http://www.lexspoon.org/blog/debian-scala-2.htmlAfter trying to dodge one potential problem getting Scala into Debian, I have now submitted a first round of packages. Two weeks later, I get word back that the packages are rejected. Why? I am told to put them in three packages instead of four.

Okay. I like my organization better, but it is not up to me. Hopefully this is the last major delay. ]]> June 17, 2006 http://www.lexspoon.org/blog/ddp-at-pldi.htmlthis PLDI paper by Manu Sridharan and Ras Bodik. They have a very interesting analysis that, from a general perspective, uses the overall approach of DDP: choose carefully where in the program you focus analysis efforts, so that your analyzer can scale to large programs but still can use context-sensitivity to unravel different parts of the program.

The authors' formulation, however, is quite different from DDP. DDP uses a very general formulation: goals, their answers, and dependencies among goals. This is so general that it is not even specific to program-analysis. This generality is a strength and a weakness: it means you can often improve an analysis simply by using DDP as the base layer of the algorithm instead of the normal demand-driven worklist algorithm. However, because DDP is so general at this level, it does not exploit any properties of the rules themselves. It is the classic trade off of generality.

One really nice property of Sridharan's graph formulation is that you can easily remove the "match" edges and thus cause the algorithm to improve its existing results. There is no analog in DDP as described so far (although I am not ready to say there cannot be such an analog).

Overall, it is great work that is very interesting to me. It is a pity the authors overlooked initially my work in the area, although we have since had a pleasant exchange in private email. More the pity is that the PLDI reviewers of this paper also overlooked the prior work. It was published in ECOOP '04, which is not obscure for program-analysis buffs. ]]> May 16, 2006 http://www.lexspoon.org/blog/patent-trolls.htmlPatent-troll companies have a business model where they acquire patents and then sue anyone who is using the patented idea. Such companies are popular to discuss lately.

Such companies are a symptom, not a problem, and they should be left alone. If a patent has been properly acquired, then its owner should be allowed to sue people who infringe on it. That is what a patent is. If someone sells their patent to another company who does the dirty work for them, then that is fine too---after all, we live in a specialized society, and not everyone is cut out to be lawyers.

The real problem is not patent-trolling companies, but bad patents themselves. In software in particular, where practically all businesses use copyright to cover their intellectual property, patents simply do not solve any real problem that we have.

In software, the best solution to dealing with patents is to abolish them. Leave patents for domains where they solve some problem. ]]> April 29, 2006 http://www.lexspoon.org/blog/exim-ssmtp.htmlIt is frustrating when programmers decide to lecture their users instead of supporting them. I just spent over an hour trying to get my mail to relay through EPFL's mail servers using the restrictive channels EPFL allows, only to find that Exim has its own restrictions. The two sets of restrictions are workable by themselves but do not have a solution together. The only way to make Exim and EPFL both happy is for them not to talk to each other. Bleh. I think I will stick with EPFL and replace Exim.

The specific constraints, for the morbidly curious and the technologically perverse:

1. Some of the EPFL mailing lists I need to use only allow mails from on campus.
2. I use this thing called the Internet, though, and do a lot of my work away from campus.
3. EPFL's IT, recognizing that 1 and 2 together are a problem, keeps #1 but adds an extra on-campus relay server to work around the problem.
4. Alas, port 25 is blocked when trying to connect to the relay server. The university firewall says no, for some reason.
5. EPFL's IT, recognizing that 1-4 are a problem together, provides a workaround for the workaround that is blocked: it lets you connect to port 465 instead of 25.
6. However, on port 465 the relay server only offers the semi-standard SSMTP protocol, not vanilla SMTP.
7. Exim, as flexible as it is, can talk SSMTP.
8. However, it will only speak it to clients, not to servers, because "ssmtp is a legacy protocol (and has been for some time)".

I wish EPFL's networking were more permissive, that it erred towards working instead of erring towards making its users work harder. I wish Exim's authors would swallow their taste, that they would support the Internet as it is instead of the Internet they wish for. I wish I had a slice of pie. ]]> April 25, 2006 http://www.lexspoon.org/blog/super-dmca.htmlAmericans should not extend the copyright protections of the DMCA. Leaving aside philosophy for a moment, just consider a couple of practical problems:

1. All computers are copying devices. On a computer, it is actually easier for the machine to copy content than to view it. Therefore, any legislation that targets copying devices, essentially targets computers in general. Everyone becomes guilty.
2. We have reached the absurd position where, for many people, it is against the law to play their own DVD's on their own DVD players. This is a basic sort of fair use that a free country should allow.

More philosophically, we should not be surprised to see the march of progress render some industries different or even non-existent. Automobiles have replaced horses and carriages, and that is perfectly normal. We should not legislate horse and carriages back into economic relevance, and nor should we clamp down on private computing just so that content providers can avoid change. Fighting progress is no reason to give up our freedom.

For more information, go visit the Electronic Frontier Foundation. Particularly interesting to me is their enumeration of fair use rights that are under threat by DMCA. We need less of this, not more. ]]> April 3, 2006 http://www.lexspoon.org/blog/debasishg.htmlblogging about Scala. Check it out, if you want the perspective of a practitioner. ]]> April 2, 2006 http://www.lexspoon.org/blog/debian-scala-long.htmlA great thing about Debian is that it welcomes anyone to become a volunteer on the project. As a result, it has over 900 developers and over 10,000 packages available.

Scala is not yet packaged for Debian. Every time I think, "all I need to do is tar up some files", I run into a discussion like this one. While Debian developers are welcoming, they are also extremely nitpicky!

In this particular case, I actually think that LAMP's release of Scala should be a "native package", as Debian calls it, which means that the source code for the Debian package is the exactly, bit-for-bit the same as the source code of the "upstream" package. That is in fact how the planned distribution would work: at the same time a LAMP release of Scala happens, there would be Debian packages created out of the same source tree. Bit for bit.

However, many experienced Debian developers seem extremely opposed to native packages, so I keep revisiting the decision. The result is that nothing happens at all. I have to wonder: is this really a big deal? The use cases cited in the above seem relatively unimportant when you consider that the absolute worst case is that someone does a diff themselves and observes that it is, in fact, empty.

Debian would be better if it had a way to upload non-nitpicked packages, especially to its unstable branch. Insisting on policy perfection before even uploading the first version means that it takes a long time to get the first version out. It is as if many Debian enthusiasts would rather talk about packages than actually build, distribute, and use packages.

As it is, Debian users who want Scala will simply have to use a generic installer. ]]> March 26, 2006 http://www.lexspoon.org/blog/debian-2006.htmlThe Debian Linux distribution is currently electing its project leader for 2006. Debian uses secret ballots, but I am going to post my votes online here nonetheless.

In general, it is a real pleasure reading the platforms and rebuttals of so many thoughtful and enthusiastic people. Software distributions are a core part of people's computing experiences, and Debian thus fills a vital role. Debian is my favorite Linux distribution, because it allows new people to contribute relatively easily, and thus it has grown to 900 developers (at last count) who maintain over 10,000 packages.

I am personally a very minor Debian developer, contributing just enough that I do in fact get a vote. While I like working on Debian, I usually have plenty of neglected research projects that can soak up any spare hours I find.

My desiderata for a project leader are:

1. Someone who thinks of Debian as producing a Linux distribution. This sounds obvious, but most of the leaders' platforms are about bureaucratic procedures. The fact that Ubuntu is making a good distribution for desktop users is a fringe discussion topic not even mentioned by many of the candidates. I favor candidates who make it the main topic. Remember the goal!
2. Debian's legal decisions are amateurish and extreme. The debian-legal list is happy to wax about law theories and legal hair splitting, e.g. the difference between contracts and agreements. At the end of this "process", they reject software that is perfectly well free for all practical purposes. This is an important subtopic of point 1: as a Linux distribution, you need to get used to the idea of distributing existing free Linux software. I would like a project leader who pushes for realistic and practical legal decisions, instead of trying to make Debian to spend its time practicing amateur law.
3. Leaders should be good leaders. For example, they should have a lot of experience with the project, not just interesting ideas. They should be good at working with people, including when they disagree.
4. The "DLT" is a bad idea. The idea is something like, a subgroup of the Debian Developers have secret meetings and then make things happen when the general group is too slow to make it happen. I love the idea of having vital subgroups. I intensely dislike giving any such group an official status, however. There should be many DLT's, not one, and they should all have to air their ideas in public before making any decisions that affect the group. This is a subpoint of point 3: Anyone who pushed for the DLT is not following good leadership.

Not a soul cares about my point 2. Debian developers of today seem happy to browbeat open-source developers for not being open-source enough. Yuck--this may motivate me to leave the project. In the meantime, the focus is on the other three:

Finally, here is my (Condorcet) vote:

1. Anthony Towns. He has been a release manager for the project. He thus has in-depth knowledge of what is in the project and what it takes to actually produce a Linux distribution (see point 1!). His writing comes off as very friendly, and the other leader nominees all say they can work with him (did I mention that Debian includes many of honest and forthright people?).
2. Steve McIntyre. His experience with making Debian CD's is positive in my accounting (point 1). An the other hand, he is a heavy proponent of a code of conduct. I admire his attention to improving communication, but I do not know if a code of conduct is the right kind of procedure to implement.
3. Jeroen van Wolffelaar. He does not have lengthy experience, but that experience is intense. Unfortunately, that experience is part of the DLT. I promote him nonetheless because he appears to be a good leader otherwise: he wants to facilitate communication, and otherwise (from my quick reading) not to do much else as the project leader.
4. Andreas Schuldei. He really likes the DLT. As good of a candidate as he is otherwise, I really think that DLT's undermine the Debian's democratic system. If he comes up with another organization for the DLT, I could happily rank him higher next year.
5. Bill Allombert. He is openly more interested in the "political significance" of Debian than the actual work of it. See my point 1. I enjoy reading Mr. Allombert's thoughts, but I would prefer for the project leader someone who focuses on getting things done.
6. Ted Walther. Ted rocks. He is very fun to read. He would be an awful leader.
7. Ari Pollak. As snarky as I am feeling this morning, I would not vote a cat ahead of a human. Shame on anyone who did. Commentary is fun and all, but actual votes should be taken seriously.

Some candidates propose applying a code of conduct around the mailing lists. On its face this sounds counterproductive to me -- a mailing list is supposed to allow all opinions, are they not? With 900 people speaking, you fundamentally need to prepare yourself to get shocked once in a while.

Additionally, many of the people proposing such codes of conduct have not been careful to say how they will enact it. Censorship (why beat around the bush?) can be helpful, sure, but it is important to be woefully careful about it if you want your organization to stay healthy.

Most importantly, though, focusing on crude comments misses the main issue. The issue is for the team to work together to enrich each others' understandings and to make decisions that everyone feels is a fair reflection of the groups' desires. Here are some directions, taken from existing democratic political organizations, that seem likely to help:

• When crafting a general resolution, Debian's closest thing to law, follow a sort of parliamentary procedure that has been adapted to the Internet. That is, everyone gets a round to speak before the next round of rebuttals comes in, etc. I am sure there are clever people in the project who could devise a good protocol.
• Branch off special-interest groups when possible so that the number of people to coordinate in a discussion is manageable. When they are done, they can bring their ideas to the general group for a larger-scale discussion.
• For yes/no legal decisions, such as those discussed on debian-legal, use a judicial system. Have someone advocating each side, and have a judge mediate the back and forth argument. The current situation on debian-legal is essentially mob rule. The popular folks simply drive through what they want and drive out people they do not like.

1. There are no subroutines or macros. If you do something twenty times, you have to repeat yourself.
2. It does not look as nice as a carefully written Latex file.
3. It does not handle math formulas, where Latex remains king.

I have long yearned for a Turing-complete language to use instead of XML. It turns out that Scala's case classes are about equally as expressive, while addressing most of these issues. The first issue goes away entirely. The second is actually reduced, but only because I wrote my own style sheets instead of relying on Docbook. (As an aside, why in the world do the standard Docbook styles generate such bad for-print versions? Latex's simple article style looks much better.) Only the third is left untouched.

Enough said. The implementations are about equally long, if you discount the style sheets. Here's the original file and counts:

• manual.xml: 795 lines, 33kb

• Manual.scala: 720 lines, 32kb
• Documents.scala: 41 lines, 1.5kb
• EmitHtml.scala: 115 lines, 2kb
• EmitLatex.scala: 138 lines, 3kb
• Total: 1014 lines, 38kb

Finally, here's the output. I think it all looks so so, although naturally I mildly prefer both of my versions to the Docbook ones.

• original: orig.html orig.pdf
• new: new.html new.pdf

In general, simplified theory is a powerful tool: it allows people to accomplish more with less mental effort. Sometimes, however, the simplification is overly strict, removing possibilities that are useful. This article focuses on an insidious assumption I think is overly restrictive: the assumption of an ever-present eval button.

The eval button is the step that converts a human-readable computer program into a computer-executable one. There is an inherent need to have some kind of divide between static program and dynamic execution. However, eval-button environments and theories make this divide absolute: there are programs, and there are executions, and the two do not mix. Likewise, if you change your mind and change the source program, then an eval-button environment makes you abandon any executions corresponding to the original version of the program.

The eval button shows up in a number of places:

• In theory, it shows up in semantics that assume the program is fixed. If someone's language semantics is summarized by E(P) = v, then this "E" function is the eval button. Such theories quietly assume that the program does not change during execution; they make it impossible to even discuss such a thing.
• In programming environments, the eval button shows up as the files+compilers arrangement and the corresponding edit-compile-run cycle. Such environments deeply assume that if you change the program, you should expect to abandon any running execution.
• In programming language design, the eval button appears as heavy static semantics. The motto of the eval-button language designer is, "If it does not type check, it is not a program". In order to achieve a programming environment without the eval button assumption, it is likely necessary that the programming language be flexible enough to allow for slop.

No Reboots!

"No Reboots!" is the rallying cry for a no-reboot programming environment. Eval-button environments insidiously presume that the running program corresponds exactly to the source code that spawned it. If the program changes, the executing process must necessarily be discarded. Stopping a program and then restarting it from scratch is nothing less than a reboot.

Example 1: Backwards Clock

I will now try, as best as possible on a web page, to show how no-reboot environments are useful to practitioners. The below examples are all done in Squeak. Demonstrating on a web page is difficult, because the interaction is the point. What I have tried here is to insert large numbers of screenshots so that the reader can hopefully picture the interaction. If it is not enough, you might also want to download David Buck's video of a Smalltalk interaction.

The first example is to modify a graphical widget. I picked at random the "watch morph" widget which displays an analog clock. My first choice for a modification was to change the display to using roman numerals, but that modification has already been implemented! As a second choice, let's make a clock that runs backwards.

The original watch morph looks like this. We start by making a duplicate of the watch morph and then telling it to be an instance of a new class. The new class is a subclass of WatchMorph that has no new methods. Thus, it begins by behaving exactly like the original. Now we can make some changes without affecting normal WatchMorph's. (Incidentally, this kind of interaction is one of the most compelling reasons, to me, to have subclassing. If the modification is planned in advance, then delegation usually works better. It is for these after-the-fact ad-hoc modifications where subclassing is really helpful.)

To succeed in the modification, we first need to understand the original class. We can open a browser on the original class and look through its methods for what needs changing. If we look at drawOn:, the method for refreshing the drawn appearance of the morph whenever necessary, we can see three calls at the top to a method called radius:hourAngle:. These three calls are trying to decide where to draw the hour, minutes, and seconds hands of the watch. The method appears to convert a (radius,angle) pair into screen coordinates.

It appears that WatchMorph is factored so that watch-like (radius,angle) coordinates are consistently transformed via the radius:hourAngle: method. To quickly test this theory, we can open an inspector on our morph and try some sample calculations. These experiments are too fidgety to display very well on a web page; I will leave you with just one example--testing out radius 0.5 and hour angle 3 (for 3:00). The feel of this is just like jiggling around a physical apparatus to get a rough feel for how it works: the information is not conclusive, but you still like being able to do it.

This radius:hourAngle: method seems to be a key modification needed for BackwardsWatchMorph. We need to change it so that the angle is interpreted in a counter-clockwise direction instead of clockwise. We can browse to the original method in WatchMorph. This needs only slight modification for the purposes of BackwardsWatchMorph. I will not post an explanation, in case you want to try figuring it out yourself. In the modified version, I have highlighted the part that changes.

This modification actually turns out to be all that is necessary. WatchMorph is already well factored to consistently use this method in all the places that a time is mapped to screen coordinates! You simply have to resize the morph in order for the morph and the system to clear some caches, and the new morph is complete.

The final things the programmer should do are to try out the new BackwardsWatchMorph from scratch, and to browse through the list of changes we have performed along the way. It is possible that the sequence of incremental changes worked, but that some kind of bootstrapping issue was overlooked. In this case, there is no problem, and the code is finished as is, but it's a good for good software engineering.

Example 2: Police Chase!

The previous example shows how a no-reboots environment can let the programmer leap seamlessly between code and object modification and browsing. Let us look at a more complicated example.

Police Chase! is a computer game where you race a police car down the highway trying to catch a bad guy. This example focuses on the police car itself. It is an interesting example because starting the game takes a minute or two while it loads textures.

Example 2a: Rate of Blinking

The police car has flashing lights on top of it. The code the causing the lights to flash on and off is straightforward. However, there is a question that is unanswerable at the command prompt: how fast should the lights blink? That is, what should the cycleTime variable be set to? This is an aesthetic question that can only be answered by trial and error.

In a no-reboot environment, it is no problem to adjust the rate as the program runs. Simply open an inspector on the car and adjust it until it looks correct. Once a good value is chosen, go back and change it in the original source code. Remember to eventually test from scratch again.

Instead of rebooting after trial flashing rate, the no-reboot environment allows programmers to delay rebooting until they are ready to test the bootstrapping code itself.

Example 2b: A Duration Class

The flashing rate is currently stored as an integer representing a number if milliseconds. Suppose we have decided that this number is used frequently enough that it is worth changing it to use an instance of class Duration to represent the time delay, instead of using an integer. That is, we want to change the type of variable cycleTime but not its semantics.

Code browsing tools can show all references to the variable. One or the other of these references must be changed first! Suppose we start by changing the definer. Instead of restarting the program, we can also change the value of the running instance using an inspector. Naturally, this last step causes a message-not-understood error. Parts of the code assume that cycleTime holds an integer, but now they see a Duration instead.

It is worth pausing a moment to consider the state of execution at this point. This is the state that eval-button proponents fear as if it were Armageddon come early. Let us take a deep breath, however, and observe the brokenness more camly--after all, broken code is part of development!

In this particular broken state, the lights of the car have stopped flashing, naturally enough. A debugger window has appeared offering a deeper look into the exact way that the code broke. Unrelated parts of the program still run, however! The road continues to flow by, as does traffic for the police car to dodge. The backwards watch morph created in the previous example continues to tick. The IDE itself is perfectly usable.

To continue onward, there is only one code patch that is necessary: the method pointed out by the debugger needs to have two calls to asMilliSeconds inserted. The system should then be told to resume executing the police car.

Missing Theory

As far as I know, no-reboot programmers operate right now in a theoretical vacuum. That is a pity. Theory should give a simplified and reliable foundation to our best practices. Let me sketch a few places where theory currently gets uncomfortable, where better theory could help us talk about and understand no-reboot environments:

• What is the semantics? In general, most language semantics do not accommodate the program changing as it progresses. We should develop fundamental semantics that account for changes in the program, including changes that invalidate portions of the existing data.
• Type checking? How should one think about type checking when the program might change after it is checked? What about the specific case where P checks, and P' checks, but P' is expected to keep executing with P's data still alive?
• Defining program changes. How should program changes be defined? Ideally, they are not defined only as changes in the static program, but also in the dynamic execution state.
• Refactoring and other nice changes. What does "refactoring" mean for changes that affect not only the program but also the execution state? Furthermore, what other well-behaved subsets of changes are worth defining?
• Scope of effects. Intuitively, some changes to a program have effects that are limited. Changing an instance variable only affects code that accesses the variable. Is there any stronger notion of effect that is worth defining and supporting?

These are just a few ideas that sketch out this rich area of research.

Summary

I have described the "Insidious Eval Button", the "no reboot" environments that avoid it, and possible theory directions to talk about no-reboot environments. Personally, I would love one day to address no-reboot environments in my own work. Meanwhile, the first step is to identify the issue and to recognize it is missing from our environments and our theory. ]]> February 9, 2006 http://www.lexspoon.org/blog/components-utopia.htmlThe components utopia is a mythical components system that allows any two components that should be able to work together, to in fact work together without any modification. Such a system is impossible, but its idea is as seductive as a perfect halting-problem detector or a perfect OS scheduler.

Components utopians say things like, different components may have been compiled against different versions of their dependencies. Therefore, any decent components system ought to support simultaneously loading different versions of the depended-on components. Non-utopians say things like, we should consider supporting multiple versions, but we are also free to say that if two components depend on different versions of another component, then those two components simply do not work together. Utopians work with a series of ultimatums in a design space that, in the end, is empty. Non-utopians admit that some components do not work together, and thereby explore a rich, non-empty design space.

It is fundamentally impossible to completely isolate two components from each other, such that they cannot possibly cause each other harm, while still allowing them to work together. Components are supposed to work with each other--after all, if I install a library, I want the other programs on my system to use that library.

Thus, you have to leave up some channels of interaction between components, and every such channel is a possibility for components to harm each other. Even speaking through a perfectly type-checked API leaves open questions like, is 0 a valid argument for this or that method? Even if you use a theorem prover on all of your components, you merely defer the problems to problems of specifying what properties, exactly, have been proven. Components that should work together sometimes simply are not going to. You have to have some mechanism for dealing with this situation and tweaking one package or the other so that they work together.

Components utopians hope that this override mechanism is not needed. They leave it out, thus causing every possible form of conflict to be a show stopper. They notice that different components have been compiled against different versions of each other, and they conclude that their system has to allow different versions of components to be simultaneously installed. They notice that sometimes they might make conflicting changes to a global namespaces, and thus conclude that you cannot have any global namespace at all. The list of possible sources of incompatibility is unending, and thus components utopians explore an empty design space. No system can possibly prevent all harms.

Therefore, a components system should include an override mechanism from the beginning! You immediately arrive at a working (though not necessarily convenient) system, because any compatibility problem can be resolved via the human override mechanism. Further, the ultimatums that components utopians face, turn into design possibilities. Instead of having to support multiple simultaneous versions of packages, one can support it. Whether to do so or not becomes a matter of trade offs. Is it more hassle to support multiple simultaneous versions, or to manually fix up the components so they can all work with one canonical "current" version? The answers to such questions are not obvious.

The package universes architecture incorporates an override mechanism based on human packaging efforts. Packages in Debian/unstable work together because humans do not post a package until they have made any small corrections that are necessary for the package to coexist with existing packages in the repository. Carefully written packages are careful with their use of the filesystem's namespace and are careful in how they update their API, and thus are installable in many Linux distributions with relatively minor repackaging efforts. Sloppily written packages are more difficult to work with, but they can still be posted -- the repackaging effort is simply greater.

The package universes architecture by itself does not specify nor rely on any of the interesting mechanisms that component system designers have devised--it only insists that there is some notion of installing and uninstalling a package. However, the architecture can very well take advantage of them. It becomes a question of optimization: Better component systems reduce the effort of repackaging, but you can get by with a poor one. ]]> February 6, 2006 http://www.lexspoon.org/blog/20060206-yahoostamp.htmlanonymous sender payment is starting to show up in email. This is the most plausible way to deal with spam I have heard of. The general idea is, you set your spam filter to be very suspicious of senders you have never heard of, possibly going to the extreme of not allowing unknown mail at all. Senders who do not make it through the spam filter can then pay a few pennies to send you email. The idea is that instead of stopping spam through direct force, you make it costly. People can still send you email you do not want, but they have to pay you.

There are plenty of clean ups possible once this basic framework is in place. For example:

• People who use cryptographic signatures can better convince an email filter that they are who they are. In turn, once they have the setup to use signatures, they can also accept encrypted email, which should probably be the norm anyway.
• There could be a universal stamping service developed, that all email services can use to accept payment. The stamps would simply be text inserted into the (encrypted) text of the email. This would enable competition between stamp providers, and interoperability between different email servers that require payment.

How should such systems be built? I'm leaning toward the following principle:

Principle: Packages of content should assume they will only be co-installed with packages from a known group of packages.

Many authors dismiss this approach quickly. However, avoiding it means that packages are intended to be loadable along with packages that were developed completely independently. It would seem inevitable, with this opposing approach, to eventually find packages that (1) expose new bugs due combination, that were not present in the packages alone, and (2) that packages will be configured reasonably but in incompatible ways with each other. Users of the RPM format are familiar with issue 2 in particular.

To contrast, the above principle does not seem onerous in practice. Debian works that way, with most of its packages being repackaged from foreign code bases. Further, following the above principle gives a number of simplifications in other aspects of package distribution, because packages no longer have to be completely bullet-proof against each other. They no longer have to insert extra indirections everywhere simply because of the obscure possibility that some other unknown package might conflict in the most horrific way imaginable.

To read more about the idea, read the Package Universes Architecture document, which describes an abstract package-distribution system based on the principle of repository-specific packages. Scala Bazaars is an evolving implementation used by the Scala open-source community. ]]> December 14, 2005 http://www.lexspoon.org/blog/20051214-unixsucks.htmlDavid Chisnall has written a good article on limitations of Unix. It is well worth a read if you want to think about the limitations of a good, mature operating system. From the thinking I've done about it, David's analysis seems quite on.

Many of the followup comments show a shallow reading that I hope is not representative. Unix is good, but David's suggestions seem accurate.

On a particular couple of points that were dismissed in the comments:

• It is great that "everything is a foo", but why does "foo" have to mean a stream of bytes? Read, write, seek? Is this the right abstraction over all OS data structures? Does it match well to sockets? TCP sockets have out-of-band data, but due to the Unix abstraction to files it is awkward to access it. How about processes and process lists? Directories in the filesystem?
• On that note, the stream-of-bytes idea is a very low-level way to interoperate, and the ubiquitous lines-of-text semistructured reperesentation doesn't work well for a lot of applications. Something like s-expressions or XML would seem to work better. Something high-level like objects might also work out pretty well.
• The X11 design decision of network transparency is pretty, but it has turned out to be a useless feature plus a heavy constraint. Almost no one uses X11 that way, because it doesn't work well that way. But because of that design decision, there is a wedge between programs and the display. The first thing you have to do to get reasonable performance is use things like Xshm to undo this wedge. This feature is expensive in performance, in design effort, and in lost features, and I hope that the next popular open source graphics framework does not make the same mistake.

Additionally, I have to disagree with types per se as the way forward. If you take Smalltalk and treated it as an operating system, then most of David's requests would be answered immediately. The answer, it seems to me, is more about using language instead of types per se. It just so happens that a big chunk of language researchers these days are type enthusiasts.

Finally, the best way forward seems to involve POSIX compatibility. Swapping in a microkernel at a low level sounds helpful, because then existing programs could run but smarter programs could use the better interaction mechanisms. Another trick that is worth thinking about is that some Unices, including Linux, now allow using /dev/fd/nnn pseudofiles as a way to convert between file names and file descriptors. File descriptors allow richer interactions between programs than stdin/stdout, but many existing programs want to see file names instead of descriptors, so this provides a way forward.

But the most important thing of all for moving forward is that the people spending their time building tools need to be aware of what is possible and what the limitations are. Posts like David's are essential to spread this awareness. ]]> December 11, 2005 http://www.lexspoon.org/blog/20051211-hos.htmlLinux distribution, possibly running on top of OS/X. It means source tarballs, makefiles, packages, and package distribution tools. It means emacs and Subversion, SourceForge and Wikis, Perl and Python and Ruby.

Not shabby. However, if you have any experience working in Smalltalk, or if you've heard the tales of the Lisp Machine, then the current state of the art pales in comparison to what we've had in the past.

In today's terms, I'd like to be able to pop open a debugger on the text editor I am using to type in this essay, make some changes, save those changes, and have them stick the next time I run the editor. If I change my mind, I'd like to invoke the version control system and undo my changes. If I like it, I'd like to be able to extract my diff, pretty it up, and send it upstream to the developers of the editor.

If I see an interesting library in /usr/lib, I should not have to do a web search in order to find its API and its documentation. My tools should know how to do that automatically. And they should do more. I should be able to ask for "all users of" the specified library, both in a static and dynamic sense, and see respectively all programs and all processes that are using the library. For the processes, I should be able to poke around at the data structures it is using and even call functions on them to see what happens.

When I want to experiment with a library, I should be able to call functions in the API without needing major effort to create a driver program to bootstrap. In fact, I should be able to play with the API by using data structures and code snippets I found via "all users of" queries. I should have a workspace, in Smalltalk parlance, whose variable references point into bits and pieces of programs, processes, and libraries all over my system.

Smalltalk and the Lisp Machine prove that all of the above is possible. In fact, given their leading examples, none of it is even seems particularly difficult.

So why are we trapped in the current steady state? Why did Smalltalk and the Lisp Machine emerge in a time when hackers were a much rarer bread, but now, when computers and CS majors are ubiquitous, hackers use something so much klunkier?

My best bet is a breach in awareness and communication between today's Linux development communities and the relative graybeards who grok either Smalltalk or Lisp Machines. Thus, when they think of ways to make a nicer user interface for Linux, they have no better ideas than to copy interfaces meant for non-hackers. These are excellent at what they do, but where is the hacker's operating system? ]]> November 11, 2005 http://www.lexspoon.org/blog/20051111-wcre.htmlIt is fun hanging out with people who toss around "3 million lines of code" like it is nothing special. For the crowd at WCRE, such large programs are the norm. These guys work with big programs, spewed out by bright-eyed coders from the distant past, that have now grown too useful to be thrown away. Dealing with these heaping piles of often-mediocre code has given the attendees a gritty, realistic outlook. Likewise, whenever these guys actually succeed with these programs, e.g. effectively reclustering the modules of a program, or translating the entire code base to a newer language, they deserve to boast a little about the accomplishment.

WCRE has the warm, pleasant feel of small conferences. There are few enough people present that you get to talk with all of them, and there are few enough papers accepted that you can attend all of the presentations. The conference itself was run admirably. Everything was close and convenient, there was free wireless Internet everywhere, the food was great, and the coffee and tea never stopped flowing.

This year the conference was co-held with WICSA, an architecture conference. The integration was quite smooth--attendees of either conference could attend the vast majority of the other one as well. The WICSA crowd also made be jealous with their nice wiki.

All conferences should have such a wiki nowadays, so WICSA is ahead of the game. A wiki is a great place for all kinds of things attendees might want to share with each other. Just a few examples are:

• continuations of discussions started after the talks or over drinks
• links to web sites that came up in conversation
• practical local information, like where is good to eat

As an example of the last item, for the first few days I kept hearing non-locals asking where to get prepaid phone cards. Apparently the in-hotel convenienc store had run out. It would have been great if folks could post locations they found on a wiki so that other attendees would know where to look.

If WCRE does take WICSA's example of setting up a wiki next year, I suggest they do not follow their lead in password management. It took me a couple of days to be able to log into the WICSA wiki. I did not previously have an account, and I had to find someone off line who had the authority to create an account for me. Just make it a wiki, already, and leave it open to editing. Either that, or have a single, bulk password that is posted prominantly around the conference. The security of restricting access to attendees only was more hassle than it is worth. Besides, it is an open conference, is it not? "Outsiders" participating on the wiki should only enrich the conversation.

On the technical material, I enjoyed the paper by Kuhn, et al., on performing cluster analysis using the text of identifiers and comments. Human language is content, too, but most reverse engineering tools ignore them because they do not have formal meanings to computers. Cluster analysis does not actually need to know the meaning of the terms, though. The analyzer can simply look for patterns in word usage.

Technical debt is a great term that James Highsmith tossed out during his Stevens Award Lecture. The concept is that before you can add new features or otherwise improve a program, you have to spend time coping with short cuts that the programmers have taken in the past. These are things I flag with "XXX" in my own code, and indeed, there tend to be embarassing numbers of them floating around if I end up rushing to make a demo.

The concept of technical debt is fascinating to think about and leaves a lot of open questions. How do you distinguish technical debt from simply programming something in a simpler way? Once you figure out the distinction, is there any way to measure how much of it you have? And whether or not you can measure it, how can it be measured?

Mary Shaw received the other Stevens Award this year. She gave a fun talk about her long career of paying close attention to uncomfortable disconnects between theory and practice. She includes many examples from her thoughts over the years, including hard-line pure functional programming and the (non-) use of proofs and formal specification to achieve correct software.

Finally, a couple of professional reengineers showed us what they do. Sneed talked about using metrics on an existing system to estimate costs of reengineering. He gets much more accurate estimates than with estimates for a new system, because the existing code gives a reliable way to understand the complexity of what is there and what will need changing. Akers talked about using automatic rewriting tools to reengineer. F