Tech Notes: 2009

Thursday, October 1, 2009

Safari discards # anchor during 301 redirect

I filed a bug with Apple a few weeks ago about Safari dropping # anchors on redirects. It is problem ID #7209106. It doesn't seem like you can link to their bugs, so I will describe it here: When visiting a URL which causes a 301 redirect, the anchor at the end of the URL is discarded. Steps to Reproduce: Visit: http://wikipedia.org/wiki/Safari_(web_browser)#Safari_4 Expected Results: The redirect to en.wikipedia.org should result in Safari visiting: http://en.wikipedia.org/wiki/Safari_(web_browser)#Safari_4 Actual Results: Safari discards the anchor, and the resulting page is: http://en.wikipedia.org/wiki/Safari_(web_browser) Regression: I did not try this with Safari 3 or earlier. But Chrome and Firefox both preserve the anchor. Notes: Here is the transaction with the server that is the redirect: GET /wiki/Safari_(web_browser) HTTP/1.1 Host: wikipedia.org User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/531.9 (KHTML, like Gecko) Version/4.0.3 Safari/531.9 Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us Accept-Encoding: gzip, deflate Connection: keep-alive HTTP/1.0 301 Moved Permanently Date: Wed, 09 Sep 2009 01:12:23 GMT Server: Apache Location: http://en.wikipedia.org/wiki/Safari_(web_browser) Content-Length: 257 Content-Type: text/html; charset=iso-8859-1 Connection: keep-alive <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>301 Moved Permanently</title> </head><body> <h1>Moved Permanently</h1> <p>The document has moved <a href="http://en.wikipedia.org/wiki/Safari_(web_browser)">here</a>.</p> </body></html>

Saturday, June 13, 2009

iPhoto 8.0.3 causes you to backup all your data again

That is, if you don't realize what happened.

I haven't found a reference as to why this was done, but after my upgrade to iPhoto 8.0.3, my rsync backup strategy decided to re-backup almost every picture I have in my library. That is until I did some poking around and found this in my iPhoto Library directory:

drwx---rwx  29 jared  staff        986 Jan 11 11:25 Data.noindex
lrw-rw-rw-   1 jared  staff         14 Jun  7 17:16 Data -> ./Data.noindex

Of course Jun 7 at 17:16 is about exactly when I upgraded to 8.0.3. Anyhow, this simple command from the appropriate point on my backup drive saved me a very large rsync:

$ mv Data Data.noindex

If you're using TimeMachine this isn't possible or desireable. I've seen forum posts on the net that imply that 8.0.3 will cause TimeMachine to re-backup everything, but it seems to me it should be smart enough to realize that those files just changed their path; the inode number probably didn't change.

Thursday, May 28, 2009

iPhone won't charge, won't connect: saved by FireWire

The other day my iPhone ran into a catastrophe: it wouldn't charge, it wouldn't connect via USB, and the battery was dying. It's a first generation iPhone, and it looked like that was the end of my old iPhone.

I followed Basic iPhone troubleshooing, read various threads about similar problems, and tried a few of my own tricks:

Tried different cable
Tried different plug
Tried different outlet
Verified that those all worked with our other iPhone
Blew gunk out of the iPhone connector area (there was actually a lot in there)
Rebooted iPhone several times
Tried to clean the tiny iPhone connector with rubbing alcohol

None of that worked. So I went to the Apple Store, and the Genius did the same things, and determined that it must be the battery. Actually, it was not the battery--the connection sound never happened, and the iPhone never recognized a connection. But the Apple Store genius was doing us a favor, because if the battery dies, we can buy an exact replacement (a first generation iPhone) for $85 dollars.

And by this time the battery in my iPhone was completely dead: I had failed to email my pictures off before the battery died. Another lesson learned...

I went home to consider my options, and try some other last resort actions, such as contact cleaner or disassembling the iPhone myself (or going to We Fix Macs). But I finally had an idea:

What if I tried to use a FireWire cable to charge?

It works!

I have an old JBL On Stage iPod speaker/dock that uses the FireWire pins instead of theUSB pins to provide power, and that worked. It still doesn't make the "beep beep" sound when you connect the iPhone (and neither does the perfectly working iPhone). That must be reserved for a USB connection. But the iPhone immediately recognizes that it's getting power, and it charges perfectly.

Note that this trick won't work with a 3G iPhone or a second generation iPod Touch: the FireWire connection inside those have been removed.

Also, I haven't yet tried to sync with a FireWire cable. I believe it just might work, but I still have to find a FireWire cable (not the speaker dock) and try.

Monday, March 30, 2009

Created Open Source project TestCpp

Today I created an Open Source project called TestCpp. It's a very simple JUnit-like C++ unit testing framework, and I'll be adding more to it in the near future.

I really wrote it because I'm working on another open source project, and I wanted to write some unit tests in C++; I quickly got frustrated with everything I had to do to get it going. With TestCpp you should be able to write your C++ unit tests and actually execute them in no time.

Wednesday, March 25, 2009

Ubuntu 8.10 and Static IP

I just installed Ubuntu 8.10 Desktop and had a very interesting time trying to configure a static IP address. There are plenty of discussions on the forums about how this doesn't just work with the standard Network Manager. And you can't just edit/etc/network/interfaces because that is ignored when you have Network Manager installed.

To make it work, I followed this procedure:

First, remove the Network Manager packages:

sudo apt-get remove network-manager
sudo apt-get remove gnome-netstatus-applet

Now you'll have to manually set an IP so that you can connect to the Internet (modify this to be appropriate to your setup):

sudo ifconfig eth0 10.x.x.y netmask 255.255.255.0
sudo ip route add default via 10.x.x.1
sudo vi /etc/resolv.conf

Set nameserver 10.x.x.z appropriately. Next install the old gnome network admin tool:

sudo apt-get install gnome-network-admin

Finally use the old GUI to set networking configuration:

network-admin

This will store the network configuration in /etc/network/interfaces where it belongs. And it seems to work when you reboot. I'll keep it this way until Network Manager is fixed.

Tuesday, February 24, 2009

Object Oriented C

In my first job, the project we worked on was 100% C code. However, it was object oriented C. This was led by our colleague Chris Westin. As we were fond of pointing out, there is a difference between Object oriented languages and Object oriented programming. You can apply OOP concepts (given appropriate primitives) in any language. Here's a table where I'll record a few of these...

Runtimepolymorphism		Achievable by using function pointers, and syntactic sugar is done with clever macros.
Link-time polymorphism		You probably do this already but don't define it as such; for instance, if you implement a function defined in a header differently on different platforms, you can consider this polymorphism. The function is different on Windows vs Linux. Another example might be a plugin for a browser. If you want it to run in Firefox and Opera, you might be able to get the core of it to call your own abstracted calls to the browser; the implementation thereof is determined at link-time.
Abstraction		This is more a matter of design than implementation, and thus is applicable to any language.
Information hiding		Again, this is a design issue. But the mechanism that is often used to achieve this is referred to as encapsulation.
Encapsulation		This can be done in different ways, but usually the most effective technique is opaque types. Again, with clever data structures and macros, this can be combined with the above Runtime polymorphism to construct objects that feel like C++, or can even co-exist with C++.
Exceptions		You can simulate some of this using `setjmp/longjmp`, but this can only go so far; the compiler doesn't know what's going on, so if you have a`try/catch` block that's really two macros doing housekeeping on the`try/catch` data structures, and then you `return` or `break` or`continue` out of the middle of that, there's nothing to stop you, and you've corrupted your `try/catch` data structures. A better method to use is to create an error data structure that can contain more information (like `__FILE__` and `__LINE__`) than a simple `int` error code. This doesn't get you the magic stack unwinding, but at least it can be more informative than -1.

Now you might be asking "why not use C++?" There are lots of answers to this, but here are a few:

Fragile binary interface problem or Fragile base class problem
This is a real problem for deployment of C++ code, and likely an important driver early on for Microsoft to develop COM. If you ship C++ objects in a shared library, you can't do so without being extremely careful about what's exposed in the header file.

Windows Debug Heap
Similarly, on Windows you must be careful with memory management. You can't cross allocations/deallocations across module boundaries in Windows, because that would be very bad. This can easily happen in C++ if you do allocations in the header file, and then deallocation in the implementation file (or vice-versa). You might be mixing memory heaps which will cause your app to crash.

Incompatible behaviors across compilers or even compiler versions.
Certainly early on different compilers or even different versions of the same compiler can generate code that is incompatible in terms of things like throwing/catching exceptions, or name mangling. This might not have been a problem for some time (I haven't checked), but is indicative of C++'s lack of an ABI.

C++ Standard library lacks ABI
Similar to the above points, if you use a certain version of compiler/C++ Standard library in your shared object, you cannot share those data types with another shared object or application that uses a different version of compiler/C++ Standard library.

C++0x doesn't appear to address any of the ABI issues that are so well known in C++. If I'm wrong, please correct me. Bjarne's C++0x FAQ (or his C++ FAQ) doesn't even mention the word "binary"; although the word "ABI" is used, but in reference to the GC system.

Lack of a "platform".
This is a common criticism, which of course C and other languages share. If you want to acquire a mutex, you have to do it differently depending on what platform you're on. Java and other more modern languages include ways to do this, and many many other things. C++0x and its standard library seem to address at least some of this...

"C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off."
Hmm... it's only harder to do in that there's more language to master.

Don't get me wrong, C++ is an extremely useful language that I use in lots of projects, but you have to know its limitations, in addition to mastering its use. I just wish that the most glaring deficiency, binary compatibility, was address in C++0x.

Exadel E7

Just attended a web conference demonstrating a new product from Exadel called E7, hosted by Brandon Blell, Charley Cowens and Max Katz.

It looks pretty cool, although I haven't played with it yet. The idea is to present to the business rules owner/author something that is removed from Java and UI code. The Java/UI author presents different types of services, such as Web services, POJO, Page services for JSF/Flex/JavaFX, etc.

Right now it only works on Seam (as Exadel is a JBoss partner).

I also asked the question "What if you had multiple UI types in your app:JSF/Flex/JavaFX. Could you set up a 'generic' type of page service so that the process can be shared across all 3 of the UI types?" The answer is that it's in development for the next release.

Also, someone asked if there's Drools integration. The answer is not yet--they're working on it for the next release.

Saturday, February 21, 2009

Comcast issues in Palo Alto

My setup at home is Comcast for ISP, and Vonage for phone service. I haven't noticed any severe degradation in my Comcast service recently... But then again, I probably don't use my home phone often, and perhaps I'm not online when outages occur.

However, that all changed on Thursday, and according to my neighbors this wasn't a one-time thing. On Thursday Feb 19, 2009, after 2pm and 4pm or so, my connection was pretty unusable. I couldn't use Vonage in a reasonable manner, and had real problems transferring data. This wasn't because of a complete inability to transfer data, it was dropping packets periodically. See the graph that I generated from dslreports.com about that time.

So I called 1-800-COMCAST and I hit the right buttons to report trouble with my "high speed Internet." A recorded informed me that there was trouble in my area, that technicians were working on it, and offered to call me back when it was better. It got better before 11pm that night, but I got my automated call back on Friday at 11am.

Some theories about what's happening:

Recently, about 2 or 3 weeks ago, Comcast made a change and blocked my incoming port 25. Ever since I've had Comcast I could never make an outgoing connection on port 25, which is "normal" for an ISP. But blocking incoming port 25 is deadly if you're attempting to run your own mail server.

The reason I mention the blocked port 25 is that I believe that these problems are related to Comcast's recent changes to control their bandwidth utilization. Comcast isnotorious for sending forged TCP control packets to upset your P2P transfers: that's like blowing out the tires of cars on a congested highway because cars might be carrying illegal contraband. In any case, they have been rightfully remorseful and punished, at least in reputation, for this behavior.

Because of this "turnaround," they've come up with a new scheme to control their traffic. Of course, every ISP has a right to control their traffic so a single subscriber doesn't swamp all other users. But perhaps their current implementation is is still a little green... And thus our current connectivity troubles.

And you can see here we (Bay Area) were scheduled for the switchover at the end of November 2008 or so, but it probably happened more recently, or there are more changes than a simple switch, and takes time to convert all neighborhoods in the Bay Area. But apparently according to this the new system is 100% online a month and a half ago.

I just did another test, and at this time it's much better, but still not great. I should probably set up a monitoring schedule with dslreports.com; it costs a little bit of money, but no big deal.

Update
I've set up the monitoring tool. Here are my up-to-date line monitoring results:
East Coast Hourly Daily
West Coast Hourly Daily

Wednesday, February 11, 2009

InterruptedIOException considered harmful

About a year ago I patched my own log4j to fix the fact that it can swallowInterruptedException via InterruptedIOException. Then I filed a bug against log4j, and got into an email conversation with Curt Arnold, who rightly pointed out that there were other scenarios where InterruptedException could be easily ignored. Anything that wraps an InterruptedException or InterruptedIOException and rethrows something that doesn’t derive from them is effectively ignoring the intended effect of a thread interrupt. The most common examples of this arejava.lang.reflect.Method.invoke andjava.lang.reflect.Constructor.newInstance which both throw java.lang.reflect.InvocationTargetException, which can have this problem.

The title of this post may be a bit misleading; java.io.InterruptedIOExceptiondoesn’t cause this problem, it just makes it at least twice as difficult, because you must always check for both InterruptedException and InterruptedIOException in wrapped exceptions. But it also means that any method that throwsjava.io.IOException must have special handlers for an interrupted thread.

I tried to find a bug in Sun’s database that warns about InterruptedIOExceptionand these cases, but the closest I could find was Sun bug 4385444. That doesn’t really have anything to do with it.

When I first saw the changes made for java.nio for interruptible IO, I thought the use of InterruptedIOException was clever and elegant. But because of the very special nature of InterruptedException, I changed my mind—of course, there wasn’t much option, because java.nio integrates with existing java.io interfaces and methods which already do not throw InterruptedException, therefore they had to follow that path. It’s really a difficult situation; it isn’t the first case in Java of a hidden or wrapped InterruptedException, it just makes it more widespread. Now you have to handle an interrupted thread anywhere java.io.IOException is thrown.

Note that on Solaris (x86 and Sparc) java.io methods can also throwInterruptedIOException. You don’t need to use java.nio to see this effect. On other platforms you only have to worry about this if you (or things you call) usejava.nio.

I’ve come to the conclusion that thread interrupts are so special, and currently so difficult to deal with, that Java should treat InterruptedException as a third type of exception: one that is implicitly thrown from every method. Of course this opens up its own can of worms, not to mention that it’s about 15 years too late to make such a change.

This also speaks to the fact that you should be following a pattern where it’s rare that you catch exceptions that you don’t understand, and should be catching them as far up as possible, where you can centralize your exception handling. I think this is also a good case for frameworks like Spring and Seam which use AoP; each method invocation can have a carefully thought out exception handler, either via your own AoP, or directly handled in the framework.

Update I’ve found bug 4176863 which is related to this issue, but more importantly, the paper Java Thread Primitive Deprecation. I had read this long, long ago, and thought it important to link here.

Thursday, February 5, 2009

Xcode templates and company name

I found a nice short description about why you get what you get in Xcode when you create a new source file:

http://symmetricdesigns.com/component/content/article/3-misc/12-changing-copyright-notice-for-new-xcode-projects-just-change-your-address-card.html

and related, how you should configure your git global settings and project settings to get the right information about you:

http://github.com/guides/tell-git-your-user-name-and-email-address

Friday, January 30, 2009

Comcast blocking incoming port 25 (SMTP)

I’ve run my own mail server in my house for quite a long time now, with no problems, no downtime, and it just works. Not anymore… Comcast has finally gotten around to my account to block my incoming port 25. As far as I can tell this started at midnight Thursday morning.

Several years ago they blocked my outgoing port 25, unless I used the Comcast MTA. That’s OK… so that’s what I did, reconfigured my postfix to use their MTA. But now that doesn’t even work—until I change it to use port 587.

A call to customer support gives you the expected response: “Are you using XP or Vista?” “You can’t read email in Outlook?” Of course, none of this is relevant. When the tech support person carries the appropriate information to the supervisor, the expected response is received: this is the policy for Comcast subscribers and there’s no option around it.

But there are still options… Here’s my list that I’ve been considering:

Plead with Comcast Has anyone had success with this approach?
Switch ISP There really aren’t many options here in the Bay Area. I’ve tried AT&T, other medium sized and smaller DSL’s, and they all have their disadvantages, including blocking port 25. But I am forever hopeful that someday we’ll get Fiosand they’ll be good enough not to do port blocking or other evil ISP things.
Pobox.com This is the service I’ve been using for 12 years now. They forward my pobox.com email address to one that I specify. Until yesterday that was an address on a machine in my closet. Now I have it forwarded to gmail. I’ve asked them if they can forward to a port other than 25, but I haven’t gotten a response yet…
No-IP This is a little different than Pobox.com. You point your MX record at their servers and they “reflect” the email right into your server with whatever address and port you give them. This costs $40 a year… The benefit over pobox.com is that I can use this for whatever email address I like with my own domain. There are other vendors, such as AuthSMTP and DynDNS (which I use for DNS), and there’s a list that’s slightly out of date here.
GMail I can just stick with GMail and be done with it. You can find lots of discussions about using GMail, or any free email service. I just would have preferred to have some control over my own data… Update: I discovered that gmail is rewriting my outgoing email address with xxx@gmail.com (this is a problem because I want everyone to remember my “permanent” address at pobox.com which is forwarded to gmail); but, you can actually teach gmail your intended email address. I saw this tip in this lifehacker article.