Wednesday, October 28, 2009 7:01:27 AM UTC 

I think now's a good time to close out the SharePoint tag on this blog, marking the end of SharePoint 2007-focused content. I'm creating this post as a sort of table of contents for my SharePoint content. I'll attempt to group posts into themes and then editorialize. Some of my earlier posts I'll even recant! Here goes.

Things I think anyone (SharePoint community or otherwise) would find interesting or useful

• Thinking Creatively - what I consider possibly my best SharePoint-related post, because it contains transferable concepts. The idea is that we as developers must go beyond our traditional code monkey role and do some critical thinking while specifying/designing solutions to problems. This is illustrated with an excellent story told during an Agile conference session. Also I recommend the linked Agile Toolkit podcast episode that inspired my post.
• The SharePoint-Python connection - if anyone ever tells you "SharePoint was written in Python", I apologize. Anyway, misquoting aside, this is a fun little bit of history.
• Golden Rule of Troubleshooting - I misdiagnosed an error, posted the erroneous diagnosis to my blog, and to save face hurriedly changed the contents of this post to be about troubleshooting. The golden rule of troubleshooting, for those of you too lazy to click through: beware the invisible proxy! It takes many forms! It strikes silently! Everyone will think you're crazy when you tell them network gremlins are eating your incoming packets, but leaving your outgoing packets alone!

Hilarity

• Welcome to SharePoint - a real-life nightmare scenario I encountered while troubleshooting a SharePoint 2003 "desktop issue." It turns out, the 15 pages of IE settings + Active Directory group policy + various Office ActiveX controls + virus scanners + IE version mix + network security appliances + Kerberos + firewalls + IE Zone settings + DNS/DHCP issues + AD replication issues + expired password issues + routing errors + spammy IE toolbars…any and all of these things, if out of whack, show the same "username/password" dialog. The post was a joke, but after troubleshooting every flavor of this problem, it gets to you a little. Anyway, welcome!
• What's wrong with this picture? - mildly amusing scenario involving disaster recovery documentation. Trust me, this is as hilarious as disaster recovery documentation is going to get.
• STSADM: Spot the typo - a lament for the endash. This is as much hilarity as you'll find on the topic of Word AutoCorrect.

Frustration

• Angry at CAML - I remember writing this post after a three days of wrestling with GetListItems, most of which was wasted learning idiosyncrasies. I then deleted most of the unhelpful angry comments, so what remains is the milder parts. This was my first "surprised by how difficult it is compared to how easy everyone makes it sound" experience. Visit for a link to the greatest Oracle DBA ever. Or visit for my graphical representation of the MSDN Rage Meter.
• I'm like, angry at numbers - in burnout mode, and ranting. If there's anything to take away from this, it's a) keep a sense of perspective (i.e. this stuff isn't important), and b) don't invest time in new Microsoft frameworks as a rule.
• Disposing SharePoint Objects: Survival Mode - this was a tipping point wherein I realized that no one does SharePoint development properly, even most of the MVPs. Keith eventually discovered and wrote down all of the disposal rules, and from there someone at Microsoft released SPDisposeCheck (which I believe covers most scenarios). Anyway, the subject of disposing is now a moot point—the more interesting bit is that, as of two full years after RTM, we had incorrect guidance as to how to dispose ~2MB objects on web servers typically running a maximum ~1000MB in the worker process. Sort of an eye-opener.
• Beyond technical challenges - rant, wherein I close the SharePoint 2007 portion of my blog (oops—the ban lasted a full month anyway, until I couldn't hold out). There are some takeaways here, notably that everyone's struggling with SharePoint, including the MVPs and "experts." I make the statement that every person working with SharePoint should look beyond their immediate technical challenge and ask, is SharePoint the right solution? Also, I challenge the assumption that SharePoint is a good developer platform.

SharePoint as an app dev platform

• [also referenced in the "useful" section above] Thinking Creatively - what I consider possibly my best SharePoint-related post, because it contains transferable concepts. The idea is that we as developers must go beyond our traditional code monkey role and do some critical thinking while specifying/designing solutions to problems. This is illustrated with an excellent story told during an Agile conference session. Also I recommend the linked Agile Toolkit podcast episode that inspired my post.
• Argue with your customer - I think I posted this after failing to convince my customer to go with less SharePoint customization and more out-of-the-box features. I still get a lot of pushback when I try to prevent SharePoint customizations. If there's something to take away from this, especially as a non-SharePoint developer, it's that not all features cost the same, and not all customizations cost the same. Making the relative costs clear to your customer should (should!) encourage them to avoid the more costly customizations. I'm always shocked at how someone will tell you "no, we need it to work exactly as I've told you!" and then turn around settle for a vendor product that does about half of what they want, but costs more.
• 80%, then stop - wherein I tell a story about my experience with the Pareto principle as it applies to SharePoint solutions. Also: you don't have to write your apps in SharePoint! If it doesn't save time, and you don't know of any benefit you'll gain from using SharePoint, then why are you attempting to use it?
• Estimating SharePoint tasks: cry for help - scary realization that I'm still unable to estimate how long something's going to take, primarily because I'm constantly blazing new (new to me?) trails in the SharePoint API, making bad assumptions about its behavior, triggering bugs, and running into unexpected limitations. Any of these things can cause multi-day delays. It does get better if you're writing a second or third app that deals with the same part of the API.

Framework limitations or errors

• How many is too many [SharePoint list] items? - the SharePoint whitepaper announcing the 2000 (now 3000) item limit per container was something of a blow. To say it clearly: this limitation prevents you from using OOB lists for anything with real traffic. There are over 3000 days in 10 years, so at 1 item added a day, you're running into the recommended limit. Since then I've seen some crazy errors with large lists, mostly revolving around OutOfMemory errors, crawl errors, using PRIME-derived features on the lists, exporting to Excel, breaking the Grid-view, and so on. So the list size limitation is real, if not a "hard limit."
• SharePoint Workflow Nuttiness, Volume 1 - My initial foray into SharePoint Workflow development ended in pain, where I had to scrap an entire approach because Workflow doesn't support state machines with replicator activites. Then I read Ayende's JFHCI series and it poisoned me forever against WF. I wonder now what problem Workflow attempts to solve, and why don't we just use a pure code solution instead? Note that Ayende wrote a book on DSLs (Workflow is a form of a DSL), so don't just pretend he's some crank with a blog*. Final note: this is one of my top 3 most visited posts, so apparently lots of people have run into the specific issue.
  *I'm aware that by definition, I'm a crank with a blog
• Illegitimate ErrorWebParts - a crazy solution to a crazy problem—here I use the "crank the chainsaw a few times" metaphor to describe loading a SPLimitedWebPartManager. Really, this is bad.
• Dingoes stole my babies! - wherein I discuss a problem with moving wiki content via the PRIME API.
• SharePoint awesomeness: User Profiles - wherein I discuss a potential benefit of functioning User Profiles. Unfortunately this post was premature, because the scenario I envisioned/laid out in the post wasn't possible out-of-the-box. Oops! Another framework limitation.
• SharePointPdfIcon project - wherein I announce my (failed) CodePlex project. It works great for single-server farms, incidentally. I just can't be bothered to spend the time to write all the timer job junk to make it work on multi-server farms when even this souped-up solution won't work when someone adds a new server to the farm after activating the Feature…this is one of those cases where using the SharePoint deployment framework causes more pain than deploying changes the "vanilla" way. Ugh.

PowerShell + SharePoint

• Find the DLLs - After determining that Lutz' (now RedGate) Reflector is a core tool for SharePoint development, the next step was acquiring the DLLs from wherever they lay. Enter gratuitous use of PowerShell to solve the problem.
• PowerShell is Magic: Part 1 - wherein I demonstrate PowerShell calling STSADM but also calculating on the fly. PowerShell is really, really useful.
• PowerShell is Magic: Part 2 - wherein I describe (poorly) how the PowerShell REPL is powerful.
• SP + PS: Working with wikis - wherein I give a pretty weak (but like the movies say, based on true events) example of how I use PowerShell to solve problems.
• Why PowerShell: Readability - wherein I take another shot at explaining how PowerShell launches processes (console apps)…and get the explanation wrong. I should probably post an update or something. Also, PowerShell can be made to be readable (though like Perl can be made to be abomination).
• PS + SP: Cornucopia - wherein I list all the real-world uses I've found for PowerShell working with SharePoint. PowerShell is uniquely useful for SharePoint, because SharePoint has a) an incomplete admin UI, b) huge object model that's loaded into the GAC, c) incomplete MSDN documentation, necessitating experimentation, and d) so much XML! Probably other reasons, but those are the big ones. Also, Visual Studio post-build tasks are the devil. I'm now ashamed of me of a year ago. Shame on you, 2008-me.
• Useful PowerShell functions I've written. I've looked at others' PowerShell functions and I think it's a lot simpler to do away with logging, comments, object disposal, and attempts to improve performance. All things are appropriate in context, but for me, these are mostly throwaway ad-hoc scripts, and are thus simple and focused.
• Write-ListDetails - particularly, discovering (and recording useful information about) large lists—and remember, this is PowerShell, so you can pull ANY data you want, no matter how complex the criteria or where the data originates.
• Run-Query - think of this as a REPL for your SharePoint Enterprise Search SQL queries. Returns pretty objects, not a DataTable.
• Get-CrawlHealth - I used this to prototype the functionality I wanted, then built it into a _layouts page. The script works though (with the exception of the $contentSource.CrawlCompleted property, which is inaccurate and worthless)
• Update-SearchScopes - on demand! You can't do this via the UI.
• Get-UserProfile and Get-UserProfileData - the first function retrieves the UserProfile object, the second function maps the (nigh-impenetrable) property collection to real properties. Useful for bulk data export and for examining your user profile data in a meaningful way.

Informational (knowledge, not concepts)

• Briefest introduction to GetListItems using CAML and lists.asmx - by now there are much better (and more accurate) guides to GetListItems. What may be amusing to you is the comments I leave on each line of code—wherein I document how uncertain I am of what each element does. The MSDN documentation has improved since 2007. As a small bonus, I'll note that this runs against WSSv2, not SharePoint 2007.
• Don't delete the default app pool - nitty-gritty details on IIS configuration. Note to anyone who has rolled out a SharePoint farm: congratulations, you're now qualified to roll out any ASP.NET app! Personally I'm pumped this knowledge transfers.
• Firefox supports auto-NTLM logins - most of you aren't aware that you can use Firefox and visit your SharePoint sites, and not be so aggravated by login boxes—Firefox supports automatic NTLM authentication in a manner similar to IE! Follow the directions to enable it.
• Cisco NLB setup in SharePoint - because I'm still the only resource for this in the entire world. Ridiculous.
• SharePoint search - faster than you think - wherein I complain about how slow IE is and how it is to blame for many of SharePoint's "performance issues." Honestly, it's true—try loading SharePoint pages in Firefox, they're way faster. Also it helps if the page doesn't load 1MB of JavaScript and another 1MB of inline style text.
• SharePoint Timer Jobs - here I attempt to shift from unhelpful ranting, to a post designed to help others avoid pain. I'm happy to say this is one of the top 3 posts, and hopefully it's helping people. Specifically, I mention that Timer Job updates require the manual reset of each Timer service on each server, and provide a script to quickly reschedule a timer job. Small footnote: I would rewrite the PowerShell script today such that it was a single function that takes arguments instead of requiring customization of the script. Functions are self-contained, and can easily be pasted into a PowerShell window (e.g. a PS window running remotely on a server!) without accidentally executing anything.
• Project retrospective on my People Search project - raw stream of consciousness, in bullet-point form. I didn't want to spend much time prettying it up, but reading the list of limitations, recommended customizations and preferred AD setup can save you weeks (and pain!) on your People Search project.

Cruft

• My SharePoint search page - to be clear, this is a static HTML page I made with search boxes to to search Google, search USENET, search the Technet forums, and search Google Reader. It's mostly broken now, and eventually I'll take it down. I used it A LOT while doing farm architecture-y type work, and used it heavily when troubleshooting in the early days. Now that I'm more development-focused, I've found I don't use it. Ever. Takeaway for everybody: Technet forums search covers more than Google does. If you're desperate enough, search both Google and the Technet forums (called MSDN Social now?).
• SharePoint search page - hottest of the hot! - wherein I add hotkey support to my (now-defunct) search page.

Op-ed (opinion pieces with almost no useful, actionable content—sorry)

• Dear MSFT, please talk to your Office division - op-ed, Sorry. Summary is, please don't obfuscate all your DLLs. Side note: InfoPath is pain.
• One Language A Year - wherein I dedicate a year to learn C#—that is, actually learn C#. I'll dig into Scala/Clojure/Haskell/Ruby/Python/Lisp/Scheme/Erlang/JavaScript/Io/Factor/OMeta/Smalltalk some other day. Also, I outright deny the claim that you should learn one language a year. It's cheap to give advice. It's not as cheap to follow advice. I have a new rule on following advice: does the person giving the advice actually do what they say? I got similarly disgruntled when "Uncle Bob" said something to the effect that you should dedicate 40 hours to work and 20 hours to learning. That's just crazy talk.
• SharePoint wikis are awesome, I swear - another of my top 3 visited SharePoint pages. I now apologize for defending SharePoint 2007 wikis. Afternote: I wish this wasn't such a popular page. Of all things, a wiki op-ed piece is one of my top pages, ugh.
• SharePoint: not unit testing - I've waffled a bit on this one. My current stance is that I'd really like to do continuous, automated functional testing (i.e. drive a browser window with code) to give me confidence my SharePoint solution actually works. True unit testing wouldn't cover enough space to give me confidence in my project, and most of my SharePoint projects are tiny, such that the "designing your API via design-by-example TDD" argument for TDD doesn't apply. Also, read this post for a short anecdotal survey on what kind of problems I run into when developing SharePoint solutions.
• Say no to makecab.exe - Here I rant against using makecab. I think I had just read yet another MSDN article that made casual use of MAKECAB.EXE and pretended like it was a good idea. Also I apparently just read the CodingHorror post on "Strong opinions, loosely held" which I now think is a terrible formula for my blog posts. At least I include a somewhat-useful PowerShell snippet that bypasses makecab, that's something.
• Surviving your first SharePoint project: Part 1 - wherein I sloppily argue that WSPBuilder is superior to STSDEV, VSeWSS, and makecab. It's true though, and somebody's got to counteract all these MSDN articles and books that pretend WSPBuilder doesn't exist…
• Does this describe you? - short, unhelpful post that quotes Niklaus Virth and laments SharePoint's accidental complexity.

SharePoint 2010/SharePoint 14 predictions

• SharePoint 14: Everything we know - it turns out from what I've heard from SPC09, this was dead-on accurate. They kept PowerPivot silent through the NDA period. Interestingly, SPC09 was silent on "Bulldog", the MDM product Microsoft purchased. Also I apparently missed out on the TownSquare bits, which they publicly discussed, and which evolved into the Facebook-like features.
• Preparing yourself for SharePoint 14 - I'm proud of my track record here, because I nailed pretty much everything. Written a full year ago.

"Other" category

• Yet another SharePoint VM: RIP - there was a period of time where I was Doing Something Wrong with my VMs. I now blame either/any of: a) saving state/restoring from saved state in Virtual Server and Virtual PC, b) running my external USB hard drive off of laptop battery power, c) lots of plugging and unplugging of said USB hard drive. I haven't had a problem in a long while now. Takeaway: back up your VM every so often "offsite", just in case.
• ASP.NET MVC is a MAGIC FLYING CARPET - wow, it's been two full years since the announcement! Anyway, here I mention how SharePoint development feels like alchemy sometimes, and separately, how the SharePoint developer community doesn't seem to value the things I like about ASP.NET MVC. Posting this had the side effect of sending lots of poor souls to my blog from google searching on "how to create an ASP.NET MVC app inside SharePoint."
• SUGDC Conference 2008: Recap - wherein I give a similarly-huge recap of each session I attended. Also: layoffs drive big SharePoint adoption! So, get with the layoffs!
• SharePoint + ASP.NET MVC - wherein I troll for people searching for these keywords.

Tuesday, October 27, 2009 3:59:44 AM UTC 

This is a two-parter. The first part is to say, hey, look at this sweet hack I've discovered in the Oxite source*! The second part is to ask, hey, is this a good idea?
* the refactored Oxite source, that is

Background services

First, let's give a little detail here—background services are long-running tasks that Oxite needs to run periodically. These are things like sending emails and sending trackbacks—necessary, certainly. But, they shouldn't be running while some chump stares at his Netscape window waiting for the site to finish sending 1000 spam trackbacks. He should be able to post to his blog, receive an immediate response indicating the post is now available, and the trackback spamming can commence later. Background services are the things you can put off, the things that don't have to finish before sending a response to your website patrons.

These background services are called by many names—I've heard cron jobs, timer jobs, background jobs, jobs, "the heartbeat," services, and tasks.  In Oxite they're called background services.

Look at this sweet hack!

The full source is below, but I'll attempt a walkthrough of the solution here. First, to explain the problem: we must achieve the impossible—we must somehow emulate a continuously-running Windows service inside an IIS worker process. This means we must periodically trigger jobs to run, but we can't monopolize valuable worker threads. And we certainly can't delay responses to send 30000 spam trackbacks. We've got to run, but we can't run anywhere in the ASP.NET page/request lifecycle! It's a conundrum.

What the Oxite team has done to achieve the impossible is, plainly, to cheat—they use a System.Threading.Timer.

How they manage the impossible is a lot like juggling—magic juggling. Enter stage left: Oxite, the juggler. Oxite takes a background task and throws it in the air. He takes hold of the next background task (let's start calling these things bowling pins) and throws it into the air, and moves on down the line. Before anyone knows what's happened, Oxite has gathered up all the bowling pins, thrown them all into the air, and made his getaway. Unlike most jugglers, Oxite makes no attempt to catch bowling pins once thrown! And this is why it's magic.

Let's try to break this back down into code. When Start() first executes [line 28], the Timer object sets a callback without halting progress [line 43]. This is the juggler throwing a pin in the air.

The callback method is eventually invoked. A thread is spun up* and runs the designated timerCallback() function [line 56]—and, let's make this clear—timerCallback() doesn't block the original Oxite web request; it lives in a new thread. And this new thread does its first dose of work, as shown on line 68 (SPOILER ALERT: it calls Run()).  We're not interested in what Run() does exactly—for today it must remain a spooky mystery, go look it up yourself.
* precisely how the thread is spun up is in fact, real magic, or might as well be to my superstitious caveman brain

Ok. Here's where the "magic" part of magic juggling comes in. Because any dunce can throw bowling pins, and any dunce can catch them, and any dunce, with practice, can juggle. The magic here is inside the timerCallback() method, where the Timer once again sets a callback. Each time a background service awakens, it does its work and, before going back to sleep, sets up the next callback with another call to timer.Change() [line 75]. That is to say, each time the bowling pin makes as if to land, it spins back upward into the air!

So there you have it. Oxite takes a bunch of bowling pins, throws them all into the air, and leaves. As the pins drop down to the ground, the "mystical Timer callback juggling force" propels them back into the air.

And we're running background threads in the web process. Sweet.

Now the question is: is this a good idea?

Now you understand how background tasks work in Oxite—or can now juggle. I get confused sometimes. In any case, congratulations!

Assuming I'm not misrepresenting anything, this is how background tasks work in Oxite. So, now for the question. Is this a reasonably acceptable way to set up background tasks for a site? I've discussed it some on twitter, but is there anything particularly nasty I've missed? Will it kill the process? Will it hang all 25 threads? Or some large portion of them?

I'm curious to hear if anyone has taken this approach, and what their experiences were.

Full source

From http://oxite.codeplex.com/SourceControl/changeset/view/45053#438025:

Tuesday, September 08, 2009 12:32:24 PM UTC 

I'm not going to get into specifics; instead I'm just going to say that my project, today, is painful to deploy. And not only is deployment painful, it's error-prone. And I don't mean "error" in the hypothetical, higher-percentage-than-that-other-hypothetical-percentage sense of the word; the sanitary, almost clinical sense. I mean, oops, there went four hours troubleshooting when I deployed the wrong DLL, preventable kind of error.

So it hurts.

There's lots of pain in the world of software development, but it doesn't have to be this bad. All I need to do is, in the beginning, set aside some time to deploy an empty shell of a project. When I say empty shell I mean, almost literally, a Hello World type of application. If this Hello World application involves seven databases, twenty seven service accounts, a network load balancer and forty web.config files, so be it. If the deployment requires granting of security permissions to these twenty seven service accounts, so be it. Sure, it's going to seem useless, and the tangible payoff will be minimal. Painful? Error-prone? Bring it on. Bring it on at the beginning.

And please, for your own sake, one-click automate the deployment! If nothing else, automate the happy path, which is orders of magnitude easier than building a fault-tolerant deployment. Worst case, your automated deployment fails and you're back to manually deploying. In other words, if you're deploying manually, then you're already living out the worst-case scenario.

And by you, I mean me, today. Again, not hypothetical.

Happy path/sad path by example: copying a folder

Happy path:

1. Copy a folder and contents to a destination directory.
2. You're done! Congratulations!

Sad path:

• Does the source file exist?
• Is the source file unlocked for read?
• Does the destination folder exist?
• Or its parent folder?
• Or its parent?
• Or the parent drive?
• Or parent network share?
• Or maybe you need to connect to the network share with a different service account?
• So this means you need to explicitly drop all current connections.
• Can we drop (delete) the existing destination folder?
• Is the folder locked?
• Is this because we have an Explorer window open to the folder (sooooooooooo common for me)
• Are we overwriting a file?
• Do we have permissions?
• Is the file locked?
• What if some of the file copy operations succeed, but not others?
• Do we have a perfect backout strategy?
• Can we restore the original folder in its entirety?
• If not, can we restore each individually changed item in its entirety?
• Are we running in a transaction?
• Are all our options atomic?
• Do we implement a transaction log of sorts? How do we know without a shadow of a doubt our operations succeeded?
• Did the virus scanner interfere when copying an .EXE?
• On your machine?
• On the remote machine?
• On an invisible HTTP proxy on the network?
• Is the remove file share something crazy like WebDAV, where only some operations are supported?
• Are you sure you're running the WebClient service required to make this WebDAV/explorer integration work?
• Are the file+pathnames reaching the maximum allowable limit, and are you copying to a deeper subdirectory which would cause "too long filename" errors to occur?

IT as a career makes me paranoid—this is a ridiculous checklist for just copying a file. But I've experienced all of these things. Yes, it's ridiculous, and yes, it's real.

Monday, August 24, 2009 10:08:21 PM UTC 

The PowerShell REPL is awesome.

PowerShell is by no means the only REPL. There's the immediate window in Visual Studio, the Snippet Compiler, LINQPad, the Interactive C# shell from Mono, and a REPL environment for most every other scripting language on the planet. Some of the TDD guys refer to "exploratory tests" that they write to learn about a third-party API. On the Regex front, there are scads of web-based and Windows-based tools to help you build and test regular expressions as fast as you can hit the "Run" button. I'll even accept writing a console application as a weak form of a REPL, though I wouldn't encourage it. All these things serve the same goal: give me instantaneous feedback. For those of you already familiar with the REPL, we're good, we're in the know.

But if you're the person who never uses a REPL, allow me to show you, using an example from just 3 minutes ago, how powerful they are.

My burning question

All this began with a burning question: what happens in string.Format() if I place the parameters out of order? What happens if I use a parameter twice?

The question, answered within 1 minute using the PowerShell REPL

Conclusion stated in words

I answered a specific question about .NET's string.Format() library function in less time than it would have taken to search and peruse the search results. Sandboxes such as these reduce the friction and enable me to run a series of experiments as quickly as I can think of them. Good REPLs (like PowerShell) allow me to a) quickly get feedback on my input commands, b) format and parse the resulting objects into a meaningful answer. Bad feedback loops (things that aren't REPLs) require overhead to even run, deliver feedback in over an hour or even as late as the next day or the next week, and deliver meaningless answers (think huge log files). I'm just here to make sure you're all aware: you have a choice: you can choose a REPL, or you can choose awfulness. Your call.

Oh yeah, what's a REPL?

Read-eval-print loop

Saturday, August 22, 2009 6:55:47 AM UTC 

Hello! I'm Peter and I'm here to present another sweet, sweet linkblog post. In my first link-heavy post, I pulled any links I could remember, from the previous week, year, or decade, a 'best of' of sorts. So don't expect great things from this sophomore effort.

To give the standard introduction: I've pulled anything tangentially related to software development in the .NET space into this linkpost, salted each link with commentary, and grouped into sections. I'm not an authority on most of the articles I link to, so when commenting on them will try to restrain myself.

Random topics

• Reminder: Virtual ALT.NET livemeetings - it's as easy as 1) typing snipr.com/virtualaltnet into your browser, and 2) entering the LiveMeeting. If you want to participate (encouraged), get a cheap mic/headset of some kind. There are three weekly meetings: Australian Monday night, Wednesday night US, and the Central-time Brown Bag meeting on Thursday. I'm definitely a fan. Also, don't forget: VAN records their sessions. Between this, NDC, and the various Ruby conference videos I've found, I'm never looking for something to watch.
• Designing Effective Dashboards [PPTX] - while I hate buzzwords as much as the next guy, and for that matter, while I hate browsing PowerPoint presentations, this has managed to overcome both its enterprisey roots and its PPTX medium! It's a PowerPoint presentation about Business Intelligence, which means I should be falling asleep just writing about it, and yet, here I am! Trust me, it's worth browsing. For a teaser, here's a self-explanatory slide:
  
  
  Business Intelligence is neat-o! Wait, did I say that?
  
  
• The circle of no life:
  
  
  Every now and again I need reminding.
  
• The Bipolar Lisp Programmer - this is partially about Lisp programmers, but also partially about programmer attitudes and, lacking a better word, 'psychology.' I don't like that word. Anyway, the article isn't necessarily based in hard science, but hey, it's a fun read, and you may just recognize something of yourself in it.
• Optimism - the three axes of optimism (or pessimism, if you're one of those people)—interesting because it provides an algorithm to make you more optimistic. Which is a good thing. And yes I said algorithm, which is why I'm linking to it.

Software engineering topics

• (Updated Classic Mistakes List) Coding Horror: Escaping From Gilligan's Island - Classic software development mistakes. You'll recognize a few (or several) from your last project.
  • Classic Mistakes Enumerated - and for more detailed descriptions of the original classic mistakes, here's the original list of 36 classic software development mistakes, copyright 1996.
• AR⊗TA - Artisinal Retro-Futurism crossed with Team-Scale Anarcho-Syndicalism - and no, this isn't a joke. This is a type of "post-Agile" thing as it tries to address the reality of failed Agile projects.

My ongoing obsession with learning, which is arguably the skill software developers need most

• Making TDD Stick: Problems and Solutions for Adopters - for those of you trying to teach others TDD, read this article for sage advice. For those of you new to TDD and frustrated by how weird and difficult TDD is, read this quote:
  
          Test Driven Development can be very hard to learn. The learning phase (the time during which it becomes a deeply ingrained habit) typically lasts from two to four months, during which productivity is reduced [2]. Eventually the benefits will be obvious and the technique is usually self-sustaining, but the question is: how to get there? Many developers give up after only a few days.
  
  …then go check out the article to see what tips they have for making learning easier.
• The 7 Phases of Unit Testing - step 1 is "Refuse to unit test because "you don't have enough time."
• We have no class (a personal journey of not learning OO) - here's some honesty about how learning object-oriented programming is a large undertaking. I'm there with you man, I'm there. I'll say that though I "get" the SOLID principles, that building systems of lots of interacting objects is messy. Procedural-based refactorings (e.g. splitting a huge method into smaller methods) are almost always straightforward and can even be measured (see cyclomatic complexity), so you know when you're on the right trail. On the other hand, when you split responsibilities into multiple classes, you get into the world of, yes, I'm going to use the word, brace yourself, here it comes: hermeneutics. Using words I understand: it means that object-oriented design is messy and mushy and it's difficult to say objectively whether a design is better or worse than competing designs. And no, I'm not going to acknowledge the pun, if you noticed, well that's your problem.
• Under Your Fingers [5 min of QuickTime video] - here Corey Haines exhorts us to engage in deliberate practice. Deliberate practice. It's an important concept, and it's something every programmer seems to be lacking. Wax on, wax off.
• The difference between derivation and innovation - Oren gives a rule which explains why so much of the new hotness in .NET is uninteresting to me. Azure, for example, is the same thing you're already doing, but in the cloud…a derivation. See, that sounds better than "it's boring" or "I hate it," right?

My ongoing obsession with TDD, unit testing, and/or producing quality software in general

• Classifying tests - good article that disambiguates unit, integration, and functional tests. Most of you reading this have an incorrect understanding of what a unit test is, and yes I'm talking to you. I'm not going to write your name here in the post, because that would just embarrass you, but trust me, it's you. So read the article :)
• A testing survey on a large project - the title isn't flashy linkbait like "40 Reasons I Can Make You Click This Link,"  but bear with me—this is one of the few places I've seen unit testing, integration testing, and acceptance/functional/UI testing put into a kind of cohesive whole. Being still in the stage where I'm forming a usual style, this type of post is great to gain a sense of perspective.
• Test Review Guidelines - Art Of Unit Testing - Roy Osherove has posted the test review guidelines from his book on this page. Most of the guidelines are unsurprising, excepting the one interesting point he makes about overspecifying tests. These are subtle points and as I struggle with this issue, it's good to read his concrete rules on avoiding overspecification.
• Unit Testing with functions that return random results [Stack Overflow] - This issue has ruined (sorry, produced valuable learning experiences during) no less than two full coding dojos, and has almost derailed a third. Now, every time we even catch a whiff of randomization, it's "uh-oh, let's break all the dojo rules and just work past the issue." So I find it satisfying to see that it's not just us.
• Evolutionary architecture and emergent design: Test-driven design, Part 2 - I'm only interested in the claim in the first section under "moist tests" that "DRY doesn't apply to unit tests." I'm suspicious that those following the "moist test" philosophy are classical TDDers and those who disagree are mockists. Yeah, I didn't make those terms up, see this Martin Fowler article. In fact:
• Mocks aren't stubs - from Martin Fowler. I found this looking for the difference between mocks and stubs, and ended up with an excellent bit of perspective: TDD advice from classical TDDers and TDD advice from mockist TDDers won't always agree, especially in the teensy details like "moist tests" above. Also, coding dojos.

Counter-counter culture (wherein we get to 'why' answers)

• The usual result of Poor Man’s Dependency Injection - Chad explains why in his experience using Poor Man's Dependency Injection (look it up if you're interested) always ends in tears. Up to reading this post, I assumed that allegiance to IoC tools was one of those irrational tribal values, but now (after reading the post) I understand.
• Why unit testing is a waste of time - the "waste of time" title is more inflammatory than the post, but the core point is that unit testing is part of a balanced diet. Part.
  Also, to be clear: unit testing is not a waste of time.

Architectures

• The Tale of the Lazy Architect - Oren describes a composable system. This sounds sweet.
• Submit this Form InfoPath - FAQs - and here is something of the opposite. This FAQ exemplifies why I don't want to ever do anything complex with InfoPath. Not to pick on Kathy the author, that isn't the point—the point is to show what kinds of awful workarounds and hacks you have to do just to make a field read-only, or to pull in a user's email address. For fairness, I should point out that this FAQ is for InfoPath 2007.

Business/organizational development/management/CEO stuff

• Culture [flash rendering of PPT presentation] - the presentation isn't very fancy but the content is dynamite. Here's an analysis of the concepts presented. I particularly like where they discuss corporate values and how the values are often meaningless ("Enron had a nice-sounding value statement"). They've got crazier (crazy in a good way) stuff in there, including a discussion of how they fire average performers. It's almost utopian-sounding.

Wednesday, May 27, 2009 1:08:01 AM UTC 

I'm here today to present the case against a particular piece of NUnit's fluent syntax. But before I do, let's set up a concrete example, something that gives the test meaning. Instead of just writing something down in boring old plain text, I've sloppily remixed a work of art I found via an image search and retitled it "Rebellion Against the (overuse of unrelated, Creative) Commons(-licensed images)!":
 

Solid. Requests for ~/Default.aspx should redirect to the Home controller. Let's get on with the show.

My Gripe: Assert.That syntax

Here is a comparison of the Assert.That syntax, the traditional Assert.AreEqual syntax, and the syntax provided by MSpec's NUnit extensions (MSpec isn't the only framework with these extensions, it's just the one of which I'm familiar):

I don't like the Assert.That syntax, in this scenario.

Look at all that. The new syntax is just…ugh.

I've read elsewhere that it's good because it reads like a sentence. Well, to shock you into elevated awareness, may I jog your memory of something else that reads a lot like a sentence? In case you didn't dare hover over that link, allow me to properly title it:

"My First COBOL.NET Web Application."

Check out the action in the linkage section!

Rock on, COBOL.NET!

Okay, conflating "reads like a sentence" to COBOL is a sucker punch, it's unfair. Let's do this by the numbers.

Comparing the three ways to do the same thing

In all cases, the fewer, the better.

Syntax	Total chars	Chars used by test syntax (by my measure)	Total times Intellisense was necessary*	Of these, times Intellisense couldn't help (e.g. Assert, Is)
Assert.That	66	28	6	2
Assert.AreEqual	58	20	4	1
result.ShouldEqual	53	15	3	0

* "Intellisense is necessary" is roughly defined as "any point at which you can use Intellisense." Definition is left purposefully imprecise.

I could go on for a bit about this, but I'll let the fancy HTML table do most of the talking. If there are any takeaways from this oversized-for-its-topic post, let them be:

• The fewer magic syntax words I have to learn (e.g. Assert, Is), the easier a framework is to learn. By this measure the MSpec extensions are the best, and the Assert.That syntax the worst.
• The fewer characters typed, the easier a framework is to use. Terseness is better when it doesn't impact learnability.
• "Reads like a sentence" is presumably the means to achieve some other goal, not a goal in and of itself. If your fluent syntax doesn't help achieve…whatever that other goal is, reconsider trying to make your fluent interface read like a sentence.

Related counter-point I don't care about today:

• The fewer  extension methods attached to "object," the better.
  NOTE: test frameworks and mocking frameworks get a pass from me because they are supposed to work against any object.

Final note: Better vs Best

I'm complaining today specifically about NUnit's Assert.That(actual, Is.EqualTo(expected)) syntax. I'm not down on the Assert.That() syntax as a whole, just the most commonly used method. And maybe that's what's bugging me—Assert.That has a lot of great stuff in there, allowing fuzzier comparisons beyond simply .AreEqual(), but the most commonly-used scenario is measurably worse than the old syntax.

And the Assert.That(actual, Is.EqualTo(expected)) syntax isn't the worst thing ever. It's not the end of the world. But shouldn't it be better than the thing it's replacing, not worse?

Saturday, May 16, 2009 11:18:50 PM UTC 

In the past I've questioned the viability of linkblogs—does anyone (including, and perhaps especially the linkblog author) have time to read all these articles?

The short answer is no. They couldn't possibly have time to read and evaluate all those articles.

I think it's become something of a cultural expectation that we scan each of the 50+ links in a daily linkblog post as a way of discovering something interesting, without having the expectation of, you know, reading anything. Inevitably the quality of the links degrade, because nobody's reading the articles. As for me, I'm batting .000 on following linkblog links this year…I'm in a kind of "linkblog hitting slump." Maybe it's just me.

This also goes for programming-related aggregators. First we had Slashdot, then briefly, Digg, then Reddit, then the front page of Reddit became something of a wasteland, so we moved to programming.reddit.com, then there was that thing called Hacker News. Somewhere along this timeline DotNetKicks reached critical mass, before slipping into the doldrums of all-ignorance-all-the-time .NET op-ed pieces; ugh. For the record I still think the aggregators do a good job, it's just that they could do better.

Where were we? Ah yes, links.

I've found the following links fascinating for some reason or other, and I personally vouch for them. If I haven't looked at the link, I'll point it out right there (which I do a lot in the "Books" section.)

Links

Podcast series (AKA Super Podcast Roundup Turbo HD Remix)
Podcasts are roughly ordered by how much I like them…but note that if they're listed here, I like them. My first podcast roundup was in 2006.

• Stack Overflow podcast - having read both CodingHorror and Joel On Software, this one's a lot of fun. Revisit old topics, get their unfiltered take on newer topics. It's good to get the unfiltered opinion, even if they're uninformed from time to time.
• Deep Fried Bytes - I like their rusty washers segment. Maybe that's the pessimist in me, but hey, I'd prefer risking listening to an over-critical rusty washers segment over over-exuberant marketing talk.
• Herding Code - when there are four co-hosts, you get better questions, and the guest isn't allowed to spout FUD/ignorance for an entire episode like sometimes happens on DotNetRocks.
• Hanselminutes - I like the recent trend of doing "follow-up" shows to correct inaccuracies on other podcast series.
• DotNetRocks - classic, still going, and like the rest of 'em, DotNetRocks has both good and bad episodes.
• Software Engineering Radio - in theory I like this show, but I'll be honest and say I haven't listened in a long while.  My commute dropped from an hour to just 8 minutes, what can I say.
• Irregularly updated podcasts I enjoy:
  • OOPSLA podcast 2008 and OOPSLA podcast 2007 - some of the best episodes/talks come from this podcast series. Hopefully we'll get the equivalent shows for their 2009 conference.
  • Polymorphic Podcast - Craig's still going, several years later. ASP.NET/web development/object-oriented development topics.
  • Elegant Code - I don't remember the last time they published something, but, hey, we're in the "irregular" section for a reason.
  • ALT.NET podcast - just switched hosts, so we'll see where this goes.
  • Rubiverse podcast - run by the former ALT.NET/now-Ruby guy. His shows are infrequent, but good.

 

Career-oriented (whether the career is freelancing, entrepreneurial, independent consulting, or even working as an employee)

• Daniel James - Building an Indie MMO (Puzzle Pirates) - this is (believe it or not) not much about making games, as it is about building a product. He explicitly mentions that you have to be extraordinarily productive. I'm not selling this well, but trust me, you'll want to check this out. Also, he wears a pirate hat.
• Archaeopteryx by Giles Bowkett - wherein he describes that he'd like to someday have Archaeopteryx (the open source app he built and loves) be his main job. Sometime late in the presentation Giles also says he'd like to describe himself as a "musician who happens to know how to program." It's an engaging laser show, fog machine and all, and as the InfoQ page says, "slides edited directly into the video since there were 500 of them." I don't agree with everything he says, but the career aspect of his presentation is something to think about.
• DHH (creator of Ruby on Rails; 37signals) at Startup School - apparently his talk immediately followed a VC who spent an hour describing how to get VC money. One of the first thing he says is "you don't need VC money" and explains why working in a VC-funded startup is like playing the lottery, instead explaining that you should follow his revolutionary advice and "charge money for your product." Engaging/entertaining, and a lot of straightforward wisdom.
• Do the Hustle, by Obie Fernandez - straightforward talk on the business aspects of independent consulting for Rails folk. Most of this applies to the rest of us.
• ajmoir's description of a hyperproductive software team - this is a Reddit conversation with multiple threads, so for the full story you've got to read all his replies. I think this is important for everyone to read because you need to believe software development can be done, for lack of a better term, "way better." The promise of hyper-productivity is fascinating. Also: Lisp.
• Mark Cuban on Success and Motivation (long, mostly storytelling)
• How to become a famous Rails Developer, Ruby Rockstar or Code Ninja - I haven't watched the presentation, but I did read his transcript. Also you may be interested in the RailsConf video feed…I haven't found anything else I'd recommend.

Screencasts/webcasts/watching presentations on your computer

• Ten Ways to Screw Up with Agile and XP - this presentation is a kind of response to the "post-agile" idea. Like the "post-agile"stuff we're beginning to hear about, he talks about how Agile projects and teams can fail. Unlike "post-agile," he doesn't blame Agile, instead focusing on solutions for the ten common problems he encounters. Highly recommended, especially for those not sold on Agile. Ben sent me this link some time ago.
• Virtual ALT.NET meetings (ongoing) - these are the in-depth presentations I've been looking for.  You can listen in live to any meeting…just plug in a working headset, go to http://snipr.com/virtualaltnet … and that's it. Also, they record sessions! Awesome! http://www.virtualaltnet.com/van/Recordings On my queue of sorts:
• Øredev 2008 videos - I'm digging through these presently. Unlike most conferences, Øredev has provided videos for each track (i.e. "the breakout sessions")! Awesome! Find the videos either
  • Split out by category ( http://www.oredev.org/ in the "Watch the videos from 2008" sidebar)
  • in one big bag of videos on VIddler
• Haven't watched - Lang.NET symposium talks. I think I'll check out the two PowerShell videos and then bail, I mean, hey—I've got plenty to check out without delving into programming language design. But, enough about me: you may find some of the other talks interesting. Big ups to Microsoft for publishing the videos.

Books (.NET development-related)

• JP Hamilton has the only "required reading" book list I've seen that's less than 4000  books. His list has three (3). I'm on 1/3. For the record, I'm okay with longer "required reading" lists, so long as they're given some sort of priority. JP prioritizes his list.
• Free PDF eBook: Foundations of Programming by Karl Seguin - in full disclosure I haven't read this. Karl doesn't directly tell you why you should read his book, but maybe this book review will help you make the decision.
• Object Thinking by David West - I'd like to think the author saw some trollish post on Usenet about object-oriented programming, and started writing a long, detailed response. One thing led to another, and hey! Object Thinking. There's a chapter in this book that has a long discussion about hermeneutics. Hermeneutics. Try and work that one into a sentence.
• PowerShell in Action 2nd Ed. is coming! Excitement! I'm biased, what can I say.

Code camps/Saturday developer events (Houston area, sorry everybody else)

• Austin Code Camp - May 30th, 2009 (soon!) - check out the hot hot hot session proposals! Hot!
• FOSS in Healthcare unconference - July 31st - August 2nd, 2009 - costs money, but maybe it's worth it to you.
• Houston Techfest 2009 - September 26th, 2009. This is the day of the Texas Tech at University of Houston game, on the University of Houston campus. Techfest: est. 600 attendees. Football game: ~30,000 (it's a small stadium). I think we'll have a crowded campus that Saturday!

Everything else

• Using Photos to Enhance Videos - this is one of those jaw-dropping demos. Click click click click click click click.
• Fred Trotter on the "VA VistA Underground Railroad" and how our US government should spend its Healthcare IT money on open source - Healthcare IT has a problem, and I hope an open source ecosystem is a solution. This article is long and gives a lot of history, so you'll get something out of it even if not interested in the politics. Also the links to the VA VistA Underground Railroad were fascinating; folks interested in Behavior Driven Development would be interested by the stories about how "a programmer sits down with a clinician" to write the app. Fascinating for a lot of reasons.
• While we're talking about BDD, you might be interested in the David Parnas' keynote at OOPSLA, wherein he lays out eerily similar goals (see the section on Documentation.)
• The underhanded C contest - 2007's underhandedly-weak encryption contest: "Your challenge: write the code so that some small fraction of the time (between 1% and 0.01% of files, on average) the encrypted file is weak and can be cracked by an adversary without the password." Make sure to look at the criteria for bonus points, and of course, the winning submissions.
• "Is anyone else here worried that they've spent so long looking briefly at everything, that they've still good at absolutely nothing?" - you don't have to click the link, just acknowledge the point. This reddit post has 1000 upvotes.
• Scott Berkun's Project management for beginners (post is short!) - because, aren't we all beginners? You don't see this kind of straightforward talk from the PMBOK (if you do, it's sandwiched between "effectively denying reality" and "having long status meetings." In other news, I think I have a rebellious attitude towards the PMI, judge for yourself.
• Abstract architecture-y type discussion - Design and Develop Versatilities, Not Applications - focus on the idea of what he calls "versatilities," and not so much on the specific technology involved (i.e. SharePoint.) I think it's a noble goal, but no, SharePoint in its current form can't realize the lofty goal he sets forth. Sorry, no. As I said elsewhere, you'll get far more mileage by training your power users to build their own SQL queries and how to use pivot tables in Excel. But the ideal is good.
• Programming Sucks! Or At Least, It Ought To - Alex (the author) runs thedailywtf.com. I don't know what to say about this. Every programmer needs to find the balance between getting real work done the ugly way, and spending time learning new techniques that make the ugliness go away. I haven't found this balance. This goes in hand with Alex's other classic article, Pounding a Nail: Old Shoe or Glass Bottle? - and carries the same assumption that you must live with your (bad) programming environment.
• All this SharePoint Stuff is Going to be Normal Soon - a lot of people see SharePoint as the next "Microsoft Web OS," i.e. that the SharePoint trend will accelerate, and that we'll start to see every future web-based product from Microsoft (and products from other vendors!) run on top of SharePoint. As it is today, the easy answer is "no that's not going to happen," because the cost of running your complex app on SharePoint can't be justified. And for tomorrow the answer still looks to be "no that's not going to happen," because I don't see any fundamental changes taking place. Non-trivial add-ons today write their data to their own database, making their "SharePoint integration" more lip service than truth. I've thought about what I'd like to see in an application framework, and if I could summarize, the one thing SharePoint doesn't support that it needs to: it would be nice if it allowed deep customizations that the product team did not anticipate. I think this is the fundamental problem which at this point is unsolvable for SharePoint. Solving this problem would require re-inventing SharePoint into something that doesn't resemble the SharePoint of today.
  But, who knows, I could be horribly wrong about all this.
• Discussion about Microsoft Gold Partners, titled "Why Your Vendor Screwed Up Your SharePoint Project" - wherein the author (gently, ever so gently) points out that Microsoft needs to change its partner ecosystem.
• How to call BS on a guru - again Scott Berkun. He writes books by the way :)
• The DailyWTF programming contest entry (a calc.exe replacement) which is built entirely in C++ templates. I can't tell you what kind of respect I have for that kind of compiler abuse.
• News: Clojure 1.0 - dismiss this at your own peril. Related: ClojureCLR alpha up.
• Is mutation testing useful in practice [StackOverflow question]? I'm reading through Kent Beck's TDD By Example, and he mentions mutation testing. Years later, it seems like no one's talking about mutation testing. Are we doing something else to test our unit tests? Is this too much overhead? Have we adopted a new mental framework that eliminates the need for mutation testing? Anyway, there's your new-old idea for the day: mutation testing.

Friday, March 27, 2009 4:48:49 AM UTC 

I'm here today to relay two messages:

First: I'm still alive and well

I haven't posted anything of real substance in quite a while (and some of you in the back of the room are shouting "in a while…or ever!" I can hear you.) I'm not here to promise more frequent and meaty updates; instead, I'm here to say that you can expect a lot less from me, at least on this blog.

My growth-as-a-developer plan (I introduced it in detail here) is going full steam. While I'm not on track to hit all specific targets, the most important thing is that I'm seeing real growth. The bits that have been most helpful for me have been a) writing my own mini-project, and b) reading source code.

Lesson: read source code!

I'd like to emphasize how drastically this has changed my outlook. First, reading others source code gives me self-confidence. And yes that's somewhat mean, I know. But it's true, and I try to beat the "you are adequate" drum as often as possible—by reading others' bad source code, you'll better know where you stand. Sometimes you realize you've got a lot to learn; sometimes you realize that hey, you're not all that bad, relatively. Bad source code can be inspiring in its own way.

And let's pull this around to the positive—I've learned a ton reading others' source code. I've picked up lots of little nuggets like using params[] as a method argument, and bigger nuggets like the several different styles of context/specification-ish unit tests. I shouldn't have to explain this; it should be self-evident that one can learn by studying source code. Duh.

Lesson: have a side project!

But more helpful even than reading others' source code is simply getting out there and writing my own. And I don't mean the type of stuff I do at work…let's not go there today. I mean code that is almost 100% logic; data stored in List<T> and passed around as IEnumerable<T>. I don't have a database. I don't have a UI. My project is entirely useless at this point, and will remain useless maybe forever.

But I'm learning a ton! What's great about building my own project is that I'm able to focus on learning specific topics. My focus points for this project are:

• OO (I'll flesh this out further when I know what it means)
• Test driven development (not just unit tests, but actual test-first, drive-out-the-design via tests, TDD)
• Context/specification-style tests

Along the way, as a kind of bonus, I've picked up:

• LINQ to objects - replacing for loops and foreach loops with LINQ calls. Related and also learning: nuttiness featuring delegates.
• Rhino Mocks AAA syntax - with the exception of method argument constraints. If someone wants to show me a good example of using constraints, I'd be ever so grateful—just a link to a project where someone's using RM constraints will work, I'll find it from there.
• NUnit/XUnit/MSpec - in that order, and yes, I switched all my tests over, and yes, the process was ugly. Also you can't claim to know NUnit if all you know is the [Test] attribute and Assert.IsTrue().

What's most important about this whole 'writing my own side project' experience is that it is fun, and I had, and continue to have, the energy to keep at it. I'm never motivated to do self-directed learning, so this boost of energy is the biggest win. If you're one of those people who can't imagine this kind of thing could be fun, well, maybe it's time to try out a side project.

Everything else has suffered

Everything else has become unimportant. Learning the newest wave of MS technology isn't even a concern at this point; I'll pick it up when I need to, or when my side project calls for it. What's surprising to me is that even ASP.NET MVC, which I happen to like, is being shunned with the rest of them.

Also, blogging has suffered. Also, my book reading has suffered.

I've completely stopped reading those futile "here's 80 things you don't know" linkblog posts. Aside from the SharePoint one, which is golden, what are you getting out of your linkblogger? Do they read all the articles they link you to? Are the links relevant/do you intend to read any of them yourself? Are the links accurate/factual? See, I'd prefer a monthly linkblogger who had on average six or seven links, and all six or seven would be interesting. Then, every year or so, there'd appear one starred link. This link would be considered so important you couldn't ignore it, a "must-see" so to speak. …Anyway, that's how I see proper linkblogging. Seven links a month, or so.

But who cares about all that, really. I'm learning a ton, and you can't stop me!

Everything in Balance

I should clarify: I'm coming to this concept as the podcast junkie/blog consumer/programming aggregator consumer person, who didn't have a side project. I've been at it (this side project) a few months now.

So if you're thinking my advice is unwise, that's fine—I'll take this space and make a disclaimer: I intend to use common sense, and re-evaluate my learning strategy from time to time. In particular, I  do intend to read books in the future, hopefully the near future.

Just not right now.

Second: I like reading my own Twitter feed

Way at the top I told you I had two things to say tonight. First was the message that everyone needs to start their own side project even just to help them learn.

Second is to tell you that I'm on twitter. Believe it, twitter.com/pseale. Subscribe! Do it!

Something I've found amusing is that I enjoy reading my own twitter feed. It's either a sign that my tweets are engaging and are chock-full of hilarity and insightful content…or that I like smelling my own aroma.  You be the judge!

Here's a sampling of my twitter bouquet:

 

There wasn't really a point to listing all these out. Well, no reason besides blatantly advertising twitter.com/pseale. Subscribe! Do it!

Thursday, February 19, 2009 5:12:52 AM UTC 

Or, how two unlike things can seem alike!

A while back, I followed a fascinating link from programming.reddit titled Pablo Picasso's version of refactoring: Reducing a drawing to 12 perfect pen strokes.

As the story goes, Pablo Picasso created a series of eleven lithographs of a bull in profile. He first created a detailed, accurate image of a bull. Then, for his next lithograph (I don't know what a lithograph is either, let's just pretend these are drawings from now on) he changed some aspects of the bull, accentuating its bull-ness. As he progressed, he began to remove detail, slowly replacing photorealism with smaller expressions of the same aspect, retaining the bull-ness. His last drawing was twelve or so thin strokes, a stick figure still roughly recognizable as a bull.

As the programming.reddit title indicated, this sounds a whole lot like refactoring!

It's super impressive, and I dearly urge you to look at the progression of Picasso lithographs yourself (click link below):

Now for the dangerous part.

.

.

.

.

.

.

.

.

.

Extra space added so you follow the link before viewing the section below; you'll miss out on the full experience otherwise!

.

.

.

.

.

.

So you're with me, right?

.

.

.

.

.

I was feeling great until I read this, the first comment on the reddit comments thread:

 

This is something with which I want to leave you. The next time someone makes a bad analogy, nail them with this Descartes quote. I can't pronounce Descartes properly, but that won't stop me, and it shouldn't stop you either. If in doubt, try a "dude, the French philosopher dude," sprinkle the word "dude" anywhere you're uncertain; they serve as TODOs for your vocabulary.

 

Aside: in true reddit fashion, this is the next highly-rated comment thread:

 

…and following that, unintentional, then intentional, references to realultimatepower.net.

Linking this discussion to the present day

This misuse of seeming similarity is (among other reasons) why a lot of us are bugged with recent CodingHorror posts. Specifically, let's take list a):

List A: SOLID principles et al

 

Here's list b), in The Ferengi Programmer (emphasis added):

List B: 285 Ferengi Rules of Acquisition

The Ferengi are a part of the Star Trek universe, primarily in Deep Space Nine. They're a race of ultra-capitalists whose every business transaction is governed by the 285 Rules of Acquisition. There's a rule for every possible business situation—and, inevitably, an interpretation of those rules that gives the Ferengi license to cheat, steal, and bend the truth to suit their needs.

And in case that was a coincidence, here's the list from his next post, responding to the standard rebuttal:

List C: processes and methodologies

 

So the question to you: are these three lists the same?

I win either way

My logic is inescapable. If you think the SOLID principles (list A) are in fact, as sneaky and extensive as the Ferengi Rules of Acquisition (list B), and are just the newest in a long line of fad methodologies (list C), then hey: I'll point you to the story about the bull, and how we all thought it was similar to refactoring. Except when you think about it, it's wasn't refactoring, it only resembled refactoring on the surface. I mean, come on, he drew pictures of a bull, it wasn't refactoring. I dare you to say the Picasso bull lithograph series was like refactoring.

And there I have you as well! Because if you refute my drawing-a-bull-isn't-like-refactoring argument, then by the very nature of your disagreement that "these two things aren't alike," you're proving that "these two things aren't alike!" Refute my "bull-metaphor doesn't apply to refactoring" argument to the "Ferengi rules metaphor doesn't apply to the SOLID principles" argument, and you've proven the very thing you're trying to argue against! I have you either way!

Next time I see you I'll collect the five dollars you owe me. And before you say to yourself "but I don't owe Peter $5," remember, my logic is irrefutable and you owe me a fiver*. Descartes says so. THE BULL! Pay up.
*this is a real word, people use it

Sunday, February 01, 2009 10:32:01 PM UTC 

This deserves its own post. After declaring that I won't be writing any iPhone apps, despite my secret dreams of iPhone app fame and riches, I went back and looked for the source of these secret, repressed dreams. Where did I get the idea that there's an iPhone app gold rush?

iPhone app gold rush stories

I didn't write the titles; the following links are as they appeared to me on either programming.reddit or Hacker News. Click each [comments] link if you're interested.

• iPhone Developer Quits Day Job After 'iShoot' Hits Number One - [comments]
• Today I lost a little faith in humanity: iPhone fart app pulls in nearly $10,000 *a day* - [comments]
• ISteam for iPhone earns a bunch of 22-year olds $100,000 in one month - [comments]
• "Crap" iPhone App Milking $200/hr - [comments]
• Developer strikes it rich with iPhone game: Makes $250K profit in two months. - [comments]

Wednesday, January 28, 2009 8:27:42 AM UTC 

I won't recap 2008; I dislike public introspection and what's more, you can read all about my 2008 by visiting my blog's home page, which has everything. I think the home page weighs in at 5MB of content right now. It's huge, and unashamed of its hugeness—my blog wears a T-shirt that says "large and in charge." The T-shirt has prominent pizza stains. Deal with it.

It's already late January, and I've missed the new year's deadline, but I'm still roughly in time for the Chinese new year.

New year's resolutions ahoy!

One thing dramatically missing from my 2008 was a proper book education. I read every programming-related aggregator known to mankind, listened to every programming podcast known to mankind, and read my share of technical weblogs. But I can't say I read programming books. Books!

I shouldn't have to explain why books are uniquely and deeply beneficial to any education. …So I won't.

Resolution: read 6 "fundamentals" books this year

6 is the reach goal, because for me, reading dense textbooks is tough. I used to put myself to sleep reading history textbooks. It turns out, Object Thinking by David West works just as well as a history textbook—even though (in both cases!) I'm interested in the subject at hand, focusing is tough.

Of the six, I'm going to start with JP's short list focused on coding fundamentals—not necessarily design, estimation, DDD, business analysis, project management, management, or whatever other useful fundamental skill you can imagine. Coding, not that other thing.

Sub-resolution: read 3 technology-focused books this year

No specifics here because I don't know which three; I'll know when I need them. I'm just writing this as an acknowledgement that yes, at some point in the next year I'll have to tackle some new frameworks; this space is reserved for three such Unnamed Frameworks.

Okay, so, books. That's obvious.

Resolution: read source code

Another obvious (and easy!) candidate is reading others' source code. Scott Hanselman has covered the why's of this topic well; I'm just here to say "me too." What's unfortunate is that I'm already running out of good samples. Most of the ASP.NET MVC samples don't even cover all the CRUD operations! CRUD!

At this one I'm doing well. So, stay the course!

Resolution: complete and release one minor development project this year

Next: practice. This is easy to describe. If I want to become a strong developer, I need to practice. Others have done a good job explaining why; I'll just say that I plan to do this. And not just attending coding dojos, which are great, but actually doing some self-directed practice. "Practice" isn't a specific goal, so instead, we'll work at one minor development project.

Minor means that it doesn't have to change the world or make me a billion jiggawatt dollars. I'm also going to try to stop reading all the rags-to-riches-iPhone-app stories that appear regularly, seducing me with their plausibility. There's been a lot of those recently (story#1 story#2 story#3 story#4 story#5 story#6).

Anyway, the point is—make a project, finish it, and do so in such a way that I'm not ashamed to release the source code. No ulterior motives, like releasing it later as an iPhone app. But if I were to release an iPhone app—I have a dream where Steve Jobs shows up on my doorstep holding a duffel bag full of cash. He's there making his daily delivery of my iPhone app's earnings. In my dream Steve Wozniak is there too, giving me a thumbs up and another duffel bag full of cash. Woz doesn't work for Apple anymore; he's there because my iPhone app is that good.

Anyway, no iPhone apps.

As a way to practice, make one project; practice techniques while making the app; no ulterior motives. Sounds easy enough. I should clarify that I can't count work projects, no matter how proud of them I may or may not be.

Resolution: boycott more Microsoft frameworks

While boycott is a strong word, it may not be strong enough to express how overwhelmed I am by the tide of technologies and frameworks coming from Microsoft! Also, it's a proven strategy—by boycotting Workflow 3.0 and LINQ to SQL in 2008, I saved a bunch of time not learning these deprecated frameworks. I'm sticking with this general strategy for 2009: if I don't need a technology, I won't pressure myself to learn it.

Putting all this in perspective

These are my technical learning goals for the year. Let me state that by no means is this my life priority for 2009. I think it would be awesome to reflect on 2009 and say "this was a great year," despite woefully failing to meet any of my stated goals above.

The point being, there are more important things than arguing about whether Silverlight matters. You know, life!

Oh and, quick, shot-across-the-bow answer: no, Silverlight still doesn't matter; don't learn it yet.

Final note: if your goal is continuous improvement, ask yourself why?

Something I noticed at the KAIZENC0NF was that there were exclusively enterprise development-related sessions (and I'm culpable as I could have suggested a topic Friday had I been there Friday). This didn't bother me at the time, but as I look back on the conference, it bothers me now. I think it's because I don't want to be truly great at enterprise development. Sure, I'm driven by a desire to be good at what I do. Sure, I want to remain gainfully employed, ideally such that I'm more valuable, rather than less, as time passes. This is all reasonable, and yes, I will put in the requisite effort. I.e., this means I'll spend time learning things I have no interest in learning, i.e. I'll work at it. The key word there being work.

But I'm not passionate about (name your enterprise vertical). I don't get excited learning a technology, framework or skill if I can only use it at work.

And don't think I just mean SharePoint (unpopular amongst .NET developers, an easy target); this applies also to the enterprise development aspects of DDD and Lean (popular, and on an upward arc*), and in learning enterprisey things like data warehouses. Or BPEL, or the abstract concepts behind BPEL. Yawn.
*the key here is to note that yes, I believe they're valuable, but no, I can't seem to get excited about learning them. Don't overreact, I just mean "I can't get excited about learning them."

What's the point of trying to become truly great at enterprise development? Just enterprise development?

Thursday, December 18, 2008 9:48:39 PM UTC 

It's all business again.

First: I'm going to be away from (hopefully all) computers for a while

I'll be on vacation, in a very real and non-metaphorical sense. Which is awesome.

What it means for you is, if on the offhand chance you leave a comment or email me, I won't respond. Sorry, I'll be away. Sucks for you, awesome for me.

Second: ASP.NET MVC and SharePoint, together as one

It still boggles the mind how many people are searching for this term. The only thing I want to ask you, the many people who are searching for "how do I get ASP.NET MVC running underneath SharePoint," is: why? Why do you want to do this?

I'm not here to provide any answers today; instead, I'm posting this as a kind of googlebait to lure you in. Maybe it's wrong, but whatever. Why do you want to use the still-beta ASP.NET MVC framework on top of SharePoint? I honestly don't know why anyone would do this. So please, if you search for "how do I combine SharePoint with ASP.NET MVC," and you hit this page, leave me a comment! I want to get in your brain and swim around a little.

I'm pretty sure I can get ASP.NET MVC running underneath SharePoint; the only magic will be removing pieces of SharePoint from the MVC project's web.config, and (maybe) integrating with SharePoint's security (or maybe not). Besides ugprading to .NET 3.5 SP1, which is by far the most arduous step on a production farm, it shouldn't be too tricky to get this working.

Anyway, there's a somewhat rambling teaser—it's probably possible, even if I can't imagine why it would be a good idea.

Thursday, November 06, 2008 8:16:47 PM UTC 
I had a moment of sudden disorientation during an elevator ride recently.

First, let me explain the elevator setup. In our fancy downtown building, we have a bank of five (or is it six?) elevators. Our elevator bank is housed in the center of the building, lining both sides of a short hallway. As fancy as we are, we aren't fancy enough to justify glass windows or any of the other elevator luxuries. The doors open, you get in, the doors close, and your new, smaller world is the four brushed-metal elevator walls.

So, as the scene had played out hundreds (or possibly thousands) of times before, the doors opened, I got on the elevator, the doors closed. This time, however, I was distracted—more so than usual—and wasn't paying too much attention to where I was walking.

As the elevator began its descent to the ground floor, and as is quite unusual for me, I had a new thought intrude—which elevator am I on? And which way do I turn when the door opens—left or right? I had no idea.

And for a brief moment, I was suddenly disoriented—almost in a physical sense.

We'll get back to the elevator story in a moment.

Conference that shall not be named so that keyword searches shall not pick it up

Last weekend I attended the open spaces event in Austin, and while I'd like to post something saying "it was a great time, well worth it, etc," I can't. There were only two impressions I have after attending the conference.

One: I'm not ready. I'm not even currently using the tools discussed by (and at times, designed by) the other attendees, nor (with my current technology stack) am I planning to use them. Tools aren't everything; my "I'm not ready" feeling also goes for the softer topics like lean/agile/kanban, which are definitely of interest to me, but not in the sense that I have any authority to make changes outside of myself. I'm not a "Big Tymer" like Manny Fresh and Baby.

Before we move onto the second impression, let me talk for a second about my learning queue, by way of Billy Hollis.

Learning queue

I listened to a fascinating Deep Fried Bytes podcast interviewing Billy Hollis. Most interesting to me was his discussion of how no one is keeping up with the .NET framework—while Microsoft is now pushing Azure and Windows 7 and C# 4.0 and whoops, throw out the old Workflow Foundation, we're pressing the reset button on Workflow 4.0—while all this is happening, of the developers Billy Hollis interviews, only ~1 out of 10 are using generics. Generics, which were introduced in 2005, and as Billy Hollis pointed out, not a large topic to learn, are still not in regular use by 9 our of 10 developers.

Sample bias noted, even if the developers he interviews aren't representative of the developer population, this is still something to sit up and take note. The key takeaway is that almost everyone is far behind. And he illustrates this with some stark (if anecdotal) numbers.

Meanwhile, over the last several years I've focused on SharePoint. I've been learning about web parts and workflow and InfoPath and web content management publishing features and ASP.NET app pools and IIS6 and XSL and Solution packages and Feature packages and governance and taxonomies and IA and so on—I've immersed myself in the SharePoint world. It was tough to keep up, especially given the magnitude of SharePoint itself.

But, at some point in the past, I publicly and officially declared, "I'm done." No more SharePoint learning, except what I need for my job, today. And it's really freed me up, in terms of mental weight. Now that I know I no longer need to learn how to do SharePoint workflow, for example, why would I ever want to learn it—especially now as they've announced WF4.0 will be completely new? Why would I want to research SharePoint object disposal best practices, when I myself no longer need this to get things done at work?

But something else happened, something unintentional. At the moment I declared I was no longer going to learn SharePoint—at that moment I experienced a similar moment of disorientation. If I'm not going to be a SharePoint guy in the long term, what now? The elevator doors will open soon; left or right?

Back to the conference

And we're back to talking about the open spaces conference I just attended. This was the conference where I was to meet up with what would become my new community of practice. This would be the group with which I could identify.

But for whatever reason, it didn't work out that way.

I've already mentioned that at the conference, I got the strong impression that I wasn't ready to attend; that I needed to do some homework before even being able to process most of what was discussed in the sessions, much less contribute.

Surprisingly, at this conference I also had a strong moment of disorientation again. Instead of cementing my understanding of software development into a rigid cast, and allowing me to fall into something of a comfortable pattern as I expected, I felt distinctly less comfortable afterwards.

I don't think it's necessarily a bad thing to be uncomfortable. If we're following the elevator story from earlier, a dubious metaphor to begin with, but hey, here we are at the end and we can't exactly go back and invent a new and possibly worse metaphor—well, let's stick with the elevator story. At the open spaces conference last weekend I experienced a kind of career vertigo—I'm in the moment just before the elevator doors opens. It's uncomfortable, but I'm sure the sensation will pass. And when it does, my world will have grown.

Tuesday, October 28, 2008 11:46:47 PM UTC 
I couldn't hold out.

It's okay though, because today we're strictly business. In the course of developing a bunch of SharePoint timer jobs recently, I've learned several things, most of which aren't obvious from the get-go:

1. Storing and retrieving configuration data is a problem. Because I don't have a farm-wide configuration list (yet—the temptation grows every day), I was forced to do some ugliness in order to store and retrieve configuration data. I don't necessarily recommend my approach; instead I'll just say I'm using a custom SPPersistedObject as my Timer Job's config store and I'll further say that it works, roughly, though I'd now prefer a better way. Consider setting up a farm-wide config list, it's a relatively big time investment but is probably worth it. Other traditional config storage options (such as the web.config, or a site-local config list, or your site-local property bag) aren't accessible to Timer Jobs without some sort of…configuration's configuration…hmm, yes…without something extra pointing the way. Anyway, it's a problem.
   
2. ANYTIME YOU UPDATE YOUR TIMER JOB CLASS (or custom assembly), YOU MUST BUMP ALL SHAREPOINT TIMER SERVICES ON THE FARM! I learned this lesson the hard way. If you fail to bump the timer service, it will blissfully run the old copy of your timer job class; I don't know how or why it caches your assembly, but it does, and bumping the SharePoint timer service is the only way to clear the assembly cache(?) and force it to use your minty fresh assembly.
1. Side-note: can we report this as a bug in the Solution framework? Because this is a big enough gotcha that the Solution framework needs to include an option to -bumptimersvc …or something. Maybe a custom stsadm command (stsadm -o resetadmsvc) or maybe tack something onto the Solution deployment API and associated stsadm commands…we need something.
   
3. There are differences between the context of a test harness (i.e. something like an NUnit integration test running under the NUnit test runner) and a timer job running in the Timer service. This may sound obvious, but when you're troubleshooting something that "only breaks on the test farm," this little bit of trivia is important. If you need to troubleshoot your timer job as it runs on the timer service, specifically, this can get tricky.
   

Also, in the course of troubleshooting just such an issue (as outlined in #3) I've created a little script to speed up the code-compile-test loop; instead of scheduling a timer job for "0100 hours" and waiting until tomorrow to see the results, why not reschedule the timer job by your own self? And that's exactly what I did.

My script below will reschedule your timer job to run 10 seconds in the future (read: instantaneously). All you have to do to get this script working for you is customize four variables to match your own timer job, then follow the quick "usage instructions" at the bottom of the script. Below is a rundown of the four variables:

• $siteUrl - the site collection root URL. We need this to get a reference to the SPWebApplication that holds your Timer Job.
• $customAssemblyName - the partial name of your custom assembly. This is necessary because we're going to new up an instance of your timer job, and thus we'll need to first load the containing assembly.
  
• $jobName - this need only be a rough equivalent of your job name. I'm usually lazy and say something like "*custom profile job*" or the minimum necessary to identify my job from all the rest. Messy is good; we're running a one-off script, right? Or you can go ahead and type the perfect exact case-sensitive job name in there, that's fine too.
• $timerJobClassName - again, we need this because we're going to new up a rescheduled timer job.
  

Assumptions:

• Your original schedule doesn't matter, and may be destroyed. Because that's exactly what this script does, by the way—it destroys the original schedule and sets a "10 seconds from now" schedule. Incidentally, whatever it did before, your job now runs on a daily schedule :)
  
• You're only concerned about the timer job running on one web application's context. Because in truth that's all that mattered to me when I wrote this script, I didn't consider the possibility of multiple jobs.
• You're running a single-server farm (i.e. a developer VM). My script only stops the service on the local server.
  
• No one else cares if you bump the SPTimerV3 service, including any other timer jobs that may be running presently. Note in the script below, PowerShell has some cmdlets to work with Windows Services. I was totally unaware of them until I had to bump this service; neat.
  

While these assumptions sound scary, trust me—you won't care. On a single developer VM, you won't care about all these things. Even on a multi-server test farm, you won't care—because this script is going to save you hours of troubleshooting.

The PowerShell script is as follows:

$siteUrl = "http://dev"
$customAssemblyName = "Corp.SharePoint.Assembly"
$jobName = "*your job name*wildcards*work*"
$timerJobClassName = "Corp.SharePoint.Namespace.TimerJob"

[void][reflection.assembly]::LoadWithPartialName("Microsoft.SharePoint")
[void][reflection.assembly]::LoadwithPartialName("Microsoft.Office.Server")
[void][reflection.assembly]::LoadwithPartialName($customAssemblyName)

function Run-Init
{
    $global:s = [Microsoft.SharePoint.SPSite]$siteUrl
    $global:webApplication = $s.WebApplication
    $global:job = $webApplication.JobDefinitions | ? { $_.Name -like $jobName }
}

function Create-NewJob
{
    Stop-Service "SPTimerV3"
    Start-Service "SPTimerV3"
    $global:job.Delete()
    $global:job = new-object $timerJobClassName -arg $webApplication
    $sched = new-object Microsoft.SharePoint.SPDailySchedule
    $now = [datetime]::now.AddSeconds(10)
    $sched.BeginHour = $now.Hour
    $sched.EndHour = $now.Hour
    $sched.BeginMinute = $now.Minute
    $sched.EndMinute = $now.Minute
    $sched.beginsecond = $now.Second
    $sched.endsecond = $now.Second
    $global:job.Schedule = $sched
    $global:job.Update()
}

#Usage: paste this script directly into a PowerShell console; the quickest
#way is to right-mouse-button click. Then when you're ready,
#run the following commands (minus the # of course):
#
#Run-Init
#Create-NewJob
#
#Anytime you update your custom assembly "Corp.SharePoint.Assembly", you will need to
#DESTROY your open PowerShell console/session and create a new one. This is the cleanest way
#to unload your old custom assembly.

That's pretty much it. Change the variables to whatever you need, open a PowerShell console, right-click, then type "Run-Init; Create-NewJob". You're done! Step 3: Profit!

Tiny footnote: if you don't care about "context", this script also allows you to execute the timer job immediately. First run the "Run-Init" function, then just type $job.Execute([guid]::Empty) in PowerShell. You can also attach to the PowerShell.exe process and do "remote debugging" of your timer job, if desired. Though if you're going to go that far, you should probably just write an NUnit test that performs the same task, and debug THAT. I'm very pro-unit testing frameworks, really, they're great. Anything that closes the code-compile-test loop, in any way, is a good thing.

Tuesday, October 28, 2008 1:52:14 AM UTC 

Yes, I'm aware it's late in the year 2008, I'm aware this stuff isn't as fresh as WPF 3D or Ruby Processing.

As I've posted earlier, I've accrued some treasured junk. Now that I have all this junk, what am I to do? Well, um…I didn't really know either.

So I started messing around.

Messing around with System.Drawing: first, infrastructure

The first thing I did was to determine the average color for a single image. I'm not sure exactly where I'm going, but I figure, hey, if you want to get a rough "picture" of what an image looks like, it's not a bad idea to look at the average color value. And we're using the RGB breakdown for color, meaning white is #FFFFFF (256,256,256), black is #000000 (0,0,0), and everything else falls in between.

Note that in my case, performance is not a big deal; I'm doing all these calculations one pixel at a time which, as you might image, is suboptimal. Mostly a straightforward operation:

public static Color Average(Image image)
{
    using (Bitmap bitmap = new Bitmap(image))
    {
        int red, green, blue;
        long redRunningSum = 0, greenRunningSum = 0, blueRunningSum = 0;
        long numPixels = bitmap.Width * bitmap.Height;

        foreach (Color pixelColor in ImageHelper.GetPixelsFor(bitmap))
        {
            redRunningSum += pixelColor.R;
            blueRunningSum += pixelColor.B;
            greenRunningSum += pixelColor.G;
        }

        red = (int)(redRunningSum / numPixels);
        green = (int)(greenRunningSum / numPixels);
        blue = (int)(blueRunningSum / numPixels);

        return Color.FromArgb(red, green, blue);
    }
}

Ok, so why do we care—it's a function, right? Well, okay, yes—but here's a PowerShell function you may also find interesting:

function Average-Images ($filenames)
{
    [void][reflection.assembly]::Loadfile("C:\a\sandbox\ImgTest\bin\Debug\ImgTest.dll")
    $i = 1
    $total = $filenames.count
    $results = @()
    foreach ($filename in $filenames)
    {
        write-host "$i - $($i*100/$total)%- $($filename)"
        $i++
        $img = [System.Drawing.Image]::FromFile($filename)
        $o = new-object PSObject
        $avg = [ImgTest.ImageHelper]::Average($img)
        add-member -inp $o -membertype "NoteProperty" -name "Filename" -value $filename
        add-member -inp $o -membertype "NoteProperty" -name "Image" -value $img
        add-member -inp $o -membertype "NoteProperty" -name "Red" -value $avg.R
        add-member -inp $o -membertype "NoteProperty" -name "Green" -value $avg.G
        add-member -inp $o -membertype "NoteProperty" -name "Blue" -value $avg.B
        $results += $o
    }
    $results
}

So. This is getting interesting. What the "Average-Images" function above does is create a custom object with some useful properties: we've got the original filename, we've got a still-breathing reference to the System.Drawing.Image object, and we're storing the "average pixel's" red, green, blue values as individual properties. The resulting objects look something like this:

Maybe it's still not interesting for you. That's fine, 'cause this party's* just getting started!
*despite what I've just written, this is not a party

I have one more piece of "infrastructure" to explain, before we can get cooking: I've created a PowerShell function called "Make-Html," which creates a permanent HTML file listing all the images I want to see, in the order I want to see them. As an added bonus, the function immediately launches the newly-created file in my browser. Here's the code:

$startDir = "C:\a\ps1\scrape\"
function Make-Html ($fullfilenames, $resultingFilename)
{
    $files = $fullFilenames | % { $_.split("\")[-1] }
    $tags = $files | % { "<div style=""float:left;""><img src=""$_""/></div>" }
    $html = @"
<html><head><title>$($resultingFilename)</title>
</head><body>
$($tags)
</body></html>
"@

    $html > "$($startDir)$($resultingFilename).html"
    ii "$($startDir)$($resultingFilename).html"
}

Ok, I know, we're still not doing anything.

Let's warm up

Okay, as I say to everyone, the real power of PowerShell is its object piping. PowerShell pipes objects, not text; this is something best seen, not heard, and hopefully we'll see a little something today. The objects we'll be slinging through the pipeline today are, as mentioned above, custom objects that have a Filename, an Image, and the RGB values representing the image's average (mean?) color.

So, let's count how many items we have:

Awesome. Let's count how many items we have that are more red than any other color:

Hmm, that was unexpected, 359 red-dominant images out of 503, that's proportionally huge. I'll point out that I did some extra fanciness to get this count to evaluate on one line, but usually (i.e. when I'm not posting to my blog) I'll work my way in parts, not all at once. So the same thing, split out, would be:

That's more realistic.

Okay, one more thing before we go. Finding out most of my pictures are red-dominant has me wondering: what about the other two? Let's work with the objects a little* to massage the answer out of them:
*a lot; ugly function that pulls out the dominant color not shown
 

Weird.

Skipping ahead to the end

This is the pattern: we'll ask a burning question, we'll form this question as a PowerShell pipeline, and we'll see the results.

Question: can we see the images in order of "redness"?

Pipeline:

$a | sort red | % { $_.filename }

Results:

Least red:

 

Most red:

Summary: okay, that makes sense. We used a naive algorithm that simply counted the red value, meaning that a pure black image or a pure blue image would have the "least redness" and a pure white image would have as much "redness" as a pure red image. Hmm, we can fix this. Onwards!

Question: Okay, so we're looking for redness. Let's call this proportional redness. Hmm, here we go:

Pipeline:

$relativelyRed = $a | select filename, @{Name="redness"; expression={$_.red / ($_.red+$_.green+$_.blue) }}
$relativelyRed | sort redness | % { $_.filename }

Results:

Least red:

Most red:

Summary: now that's more like it. Our earlier naive results were instructive, but this is more what I was looking for.

Question: okay, so let's stop messing with redness. Instead, let's find out what images have the most variance between the colors. We're less interested in the white-gray-gray-gray-black spectrum, and are looking for more colorful images. Let's do this:

Pipeline:

$variance = $a | select filename, @{Name="Variance"; Expression={$avg = ($_.red+$_.green+$_.blue)/3; $var = [math]::Abs($_.red-$avg) + [math]::abs($_.green-$avg)+ [math]::abs($_.blue-$avg); $var} }
make-html -fullfilenames ($variance | sort variance | % { $_.filename }) -resultingFilename "variance"

Results:

Most balanced:

Most variance:

Summary: most interesting, besides a grouping of the "grayish" and "black and white" images all together, is the smattering of images that have color, but are so perfectly balanced they're nestled right in there with the pure black-and-white images. Neat.

Final bits

This post is already too long. There's not too much else to say, besides a) stuff is awesome, and b) with the aid of either PowerShell functions or .NET library calls, you can do some complex things. If you only remember one thing from this post, try and pick up the impression I'm trying to leave. This is how I see PowerShell: it's an experimental playground where I morph a thought, an idea, slowly into something workable, and in each step along the way, I'm getting feedback and refining, and in the end, I've satisfied my curiousity. Maybe it's something as useless as basic image analysis using System.Drawing.

Incidentally, if you want to see how the professionals do this kind of thing, check out Multicolr - an color search engine indexing 10 million Flickr pictures, which makes the stuff I did above kind of pitiful looking :) When I checked last, the Multicolr site was slow, otherwise it's neat; check it out.

Tuesday, October 21, 2008 11:58:07 AM UTC 

HOWDY!

This is a quick announcement to let you know that my site now runs on ASP.NET MVC. A few things have been updated:

• My sweet SharePoint search page (follow this link for description) has been updated with the NEW MSDN forums search, and the NEW Google Groups (USENET) search. It turns out I wasn't searching USENET much because this thing has been broken for months now . Anyway, check it out—you either need it or you don't.
• My best-of-breed CAPTCHA is back! See pseale.com/captcha for details.
  
• Similarly, my regularly-updated wealth clock (pseale.com/clock) is back! I'm sure you all missed it as I have.

(strike that) dasBlog 2.1 running on ASP.NET MVC Preview 5

dasBlog 2.2 running on ASP.NET MVC Beta

First thing I should point out is that I'm running under IIS7 Integrated mode on shared hosting. If you're attempting this, be sure you're running on IIS 7 in Integrated mode. If you're trying to test this out on your own machine, this means you must be running Vista or Server 2008, must create a fresh web site in IIS and make sure the app pool is running on Integrated mode. Let me be clear: you can't properly test on the Cassini web server running your Visual Studio project.

ALSO: Now that ASP.NET MVC Beta adds itself to the GAC, for a while (until you host loads the dll's to its server's GAC), you'll have to make local copies of each ASP.NET MVC dll. "Copy Local" a property under each assembly Reference—set it to True for each one of them.

Ok. There are three things you have to do to get dasBlog working underneath an ASP.NET MVC app.

First, in your MVC app, set Routing to ignore your blog's folder. Mine is called "/blog". Here's what it looks like (I think I stole this from a Phil Haack blog post, so if it looks familiar, it is):

 

Second, and this won't be an option for all of you—I removed all the System.Web.Extensions (AKA ASP.NET AJAX, AKA "Atlas") from my root MVC app. This fixed the problem I was experiencing with my ASMX-powered RSS feed, which Atlas usurps by default (thanks Ben for the tip, that did the trick).

Third, we need to do some heavy work on the dasBlog web.config. First I'll say, thanks to Paulb on the dasBlog team for providing the starter IIS7 web.config. Big ups to changeset 14700.

Instead of attempting to explain in detail any of the nasty things I've done to make the dasBlog 2.2 web.config work under the most recent MVC drop, I'll just post my web.config directly for viewing. I don't recommend what I've done for others; instead I'll say that I got my web.config minimally working underneath a small MVC-based site.

If you're reading this blog post because no one else has provided a better explanation, then maybe perusing my web.config will help.

Without further ado, I present you: web.config of my dasBlog application running underneath an ASP.NET MVC site.

KNOWN ISSUE: http://www.pseale.com/blog (without the trailing slash) bombs out with an error. http://www.pseale.com/blog/ works fine. I assume it has something to do with the blowery.web component, something that I have no desire to fix; I'll work around the problem with a Routing fix/hack. Anyway, lesson learned: BEWARE TRAILING SLASHES!

UPDATE: apparently the blowery.web compression is somehow interfering with delivery of my CSS files. I say apparently because I didn't attempt to troubleshoot this, I just disabled the HttpModule…with extreme prejudice! As much extreme prejudice as one can muster against an HttpModule, anyway.

Tuesday, October 14, 2008 3:31:12 AM UTC 

 

Recently I've been trawling THE INTERNET for retina-dissolving or otherwise awesome images, and have programmatically collected/mushed them into the nuttiness above. More to follow soon, unless I'm lazy. So, uh, probably more to follow…eventually.

EDIT: reposted with an image that is NOT 3MB. Yes, the original image was 3MB, a catastrophically large file. I'm like the guy who sends a holiday greeting PowerPoint over email that brings down the mail server for two days. Thankfully no one subscribes to my blog, otherwise that could have created "heap big bandwidth bill." I blame Windows Live Writer and Paint.NET, daring me to paste directly from one program to the other. For shame, Paint.NET. For shame.

Tuesday, September 23, 2008 3:10:01 AM UTC 

Summary first, because I just re-read this and it's ridiculous long

If you've subscribed to my blog exclusively for the SharePoint bits, sorry; now is the time to unsubscribe.

That goes for both of you/half my subscriber base :)

I've become increasingly frustrated with SharePoint (I'll explain why in, um, detailed fashion), and instead of souring like a lemon, I'm just going to refocus. This means I'm dropping out of the SharePoint blogosphere to focus on long-term fundamentals; knowledge that won't expire when SharePoint 14 hits beta. I may jump back on for SharePoint 14, but it all depends.

UPDATED 2008-10-18: fixed grammatical and factual errors. By factual errors, I mean that time when I attributed something to Chris O'Brien's blog, when oopsies! it wasn't him at all.

Introduction

I've become increasingly frustrated with SharePoint recently (again, I know), but what's been bugging me more lately is the fact that I'm solving all the wrong technical challenges. I think I have a (deeply suppressed) aesthetic sense, that every so often, rears its ugly head. Or flares out from my neck like a humongous goiter. Goiters are a serious problem—hey!—used iodized salt, it's that easy.

I get agitated at times staring at (undocumented) CAML, or anytime I work with InfoPath, or remembering when to surround hardcoded GUIDs with curly braces (and when not), or knowing when to dispose SharePoint-created objects, or remembering which Workflow activities only work in SharePoint Designer and not Visual Studio, or working around concurrency bugs, or troubleshooting failed Solution deployments. And so on, pick your topic; these are real issues by the way.

So what's the point of this post again?

I want to send the message out there to everyone working with SharePoint, to look beyond your immediate technical challenge, and think. Think beyond your immediate issue, beyond with what you're wrestling at this particular moment, beyond your immediate (sometimes overwhelming) technical challenge with me. Are you ready? Looking at the size of this post, maybe not!

Let's DO THIS.

Are you an expert at SharePoint development?

I recently watched Dave Thomas give a talk about developing expertise. It's interesting, go watch it if you have time; he's not making it all up on the spot, he relies on some kind of research. In the talk he does something I like: he categorizes everyone (using the Dreyfus model) somewhere along the expertise scale: Novice, Advanced Beginner, Competent, Proficient, Expert. And gives definitions for all five categories, and gives helpful tips for how to work with people of varying skills. It's not all fluff, go watch it.

Bringing this back to SharePoint: after trolling the entire SharePoint blogosphere for quite a while now, I've come to the conclusion that there are no expert SharePoint developers. I'd go as far as to say that there are very few proficient SharePoint developers, and you probably know all their names (hint: look for "MVP" in title and/or multiple Bentleys in the parking lot).

The rest of us are just struggling. The "everyone's struggling" theme really sunk in recently when I browsed the source code for SharePoint-tagged CodePlex projects. All of them (save one) disposed their objects improperly. Including mine, to be fair, I'm not hating on your free, labor of love project, I'm making a point. The point isn't to hate on your baby, the point is to say that no one is doing SharePoint development properly. Look on CodePlex; you'll see what I mean.

Let that sink in. No one is doing SharePoint development properly. And hey, your team may have tackled all the object disposal issues, but maybe instead, you're ignoring large portions of the framework and (unknowingly) rewriting large portions of it. Maybe you don't bother packing things in Features and Solutions; maybe you spend too much time packing things in Features and Solutions. Maybe you're great at writing Site Definition CAML but haven't gotten the memo that putting everything in your site definition is a bad practice (or the other memo telling you that custom Site Definitions don't upgrade to v14). Maybe you're unknowingly triggering framework bugs that ruin your customers' trust in your solutions. Maybe you write unmaintainable code.

Maybe you don't understand your customer's core problem!

Maybe, in your metaphorical dwarven greed, you've delved too deep into the framework, and unknowingly stirred the framework Balrog. And you don't want to wake the framework Balrog, let me tell you.

I don't know about you, but I'm there—I'm struggling.

Takeaway: just try to be competent

Dave Thomas says in his expertise talk that advanced beginners think they're much further along than they are, and those who are proficient or experts think they're beginners. Let me tell you: as far as SharePoint development skills go, I'm an advanced beginner, bordering on competent. And I mean it—knowing that he says "beginners tend to think they're experts, and experts think they're worse than they are"—with all that said and digested, I still think I'm a beginner (advanced beginner, I'm not a total scrub :) ).

So, and this applies to those of you who are still new to the SharePoint world and/or isolated—it's okay to admit that you're not an expert. If you need an ego boost, go look at CodePlex—you'll certainly learn a great many things from others' code, but you'll also realize that hey, these people don't have it together either!

Becoming a competent SharePoint developer

What I've also begun to realize, is that the more I learn about SharePoint, the less I want to apply it to different scenarios. Where before I used to say "SharePoint Workflow is supposed to be good at solving that problem," now I say "SharePoint Workflow is painful, and for long-term maintainability I'd recommend you manually code up whatever you need, elsewhere." Whereas I used to say "InfoPath is useful for simpler scenarios," now I further limit InfoPath to "end-user tool only; don't you dare use InfoPath's code-behind." I say these with specific scenarios in mind, I have biases, your mileage may vary, all things within reason, etc, but the point I'm making stands—I'm recommending SharePoint less.

Dave Thomas says, again, that as an advanced beginner you believe that nothing works. Unfortunately, every time I encounter a new piece of the SharePoint framework, I'm an advanced beginner. I don't have the opportunity of being the CQWP guy, every day. Today I'm that guy, tomorrow I'm the InfoPath guy, day after I'm the admin, day after I'm the help desk. I'm that guy.

And hey, I've been an advanced beginner before. For example, I distinctly remember the first time I tried to do Active Directory scripting with PowerShell. It was painful. But gritting my teeth and working through the pain, I became comfortable with it. And, now that I know my way around, I'm able to do some cool (simple) things with PowerShell and Active Directory.

This is a common pattern—I've been an advanced beginner elsewhere, worked through the pain, and by the end, I was comfortable and was feeling more confident.

What I'm trying to articulate in these long rambling paragraphs, is that as I learn more about SharePoint via experience, I become comfortable with pieces of the framework and trust the framework less. As I learn more, I trust SharePoint less. Today I'm totally comfortable in telling you that InfoPath is absolutely great for simple data entry, for simple review scenarios, but nothing else*. And this isn't just InfoPath. As I work my way around the framework, I am becoming comfortable with individual pieces, and as I do so, I am trusting each piece less.

And let me set the record straight. I'm less interested in whether a particular technology is Turing-complete—whether it can theoretically solve any problem—I'm more interested in whether using that technology is a good idea in the first place. And as I learn more about SharePoint, the world of good ideas seems to shrink.
*note for InfoPath nitpickers: yes, it's useful for other things, but yes, it's also easier for me to say "good for nothing else" than rattle off 15 edge cases.

Do you want to become proficient?

I think what is bugging me most is that I don't want to be a SharePoint apologizer. Not apologist, apologizer.

Oh, sorry, that doesn't work. Sorry, yeah, there's a bug. No, yeah, you heard it was great for ECM; well, yeah, it's cheaper, but we'll have to customize a lot. Oh, yeah, that's undocumented. Oh, no, we hit the list size limit early. Yeah, no, SharePoint Designer is unmaintainable, don't build solutions with it. Oh yeah, factor in cost of upgrades on every SharePoint project.

Last week we found out "the hard way" that SharePoint can't handle 12000 documents in a single folder, no metadata, no versioning, nothing fancy. Performance whitepaper aside, 12000 documents shouldn't be a problem, I don't want to hear about it—this is a bug. And meanwhile, from the other end I'm attempting to fend off a total migration of terabytes of network file shares to SharePoint, because "this should work, right?"

I don't want to progress with SharePoint, if my energy is going to be spent tackling the wrong challenges.

Add in the cost of learning

Something that I'm maybe not emphasizing well enough is the point that in theory, all things are possible in SharePoint. And if the cost of becoming a SharePoint expert was zero, SharePoint development would be no question, worlds better than coding by hand. No contest, no question, better.

But learning isn't free.

SharePoint gold rush

Dux Sy refers to a SharePoint gold rush. It's real. It's real in part because the time cost of learning SharePoint is so high. Chris O'Brien (I can't find the entry on his blog) Shane Young states that you should be wary of hiring developers if less than 5 of their last 10 projects are SharePoint projects. It's expensive to find people with that kind of SharePoint experience. Thus, high prices, thus, gold rush.

This equation works for all Enterprise solutions; when I talk about how SharePoint is aggravating my inner engineering brain, trust me, I've worked with other Enterprise systems, and they're worse. SharePoint's a great "Enterprise" product—believe it or not, Microsoft is quite open compared to other vendors, who require all independent consultants be licensed by the vendor, or who provide no public documentation at all—all these extra hurdles artificially inflating consulting prices. Microsoft thankfully doesn't do these things (at least that I notice). So, it could be worse.

No more SharePoint on my blog

I've posted exclusively SharePoint content for a while now, and as I do so, I notice my attitude continues to trend negative. It's my doom-and-gloom engineering brain, it can't just let it go. So, instead of ranting further, I'll just, hmm, let it go. No more SharePoint 2007-related topics on my blog, even if a potential topic is brilliant (WHICH IT INEVITABLY WOULD BE!). No more SharePoint content unless I just can't hold it in.

If you've subscribed to my blog exclusively for the SharePoint bits, sorry; now is the time to unsubscribe.

I'm also officially done learning SharePoint 2007 development, permanently—no more off-hours learning, no more digging through my humongous SharePoint blogroll. When SharePoint 14 is announced, I'll evaluate it to see if they've fixed anything—to see if the situation's improved any. If so, great! Maybe SharePoint 14 will realize the potential of what some are calling "the first real application development platform from Microsoft." There are some awesome business problems that SharePoint already attempts to solve—that with just a little extra help, SharePoint could be a powerful solution. It could be dominant, even.

But if SharePoint 14 is more of the same; if they announce SharePoint Designer Designer 2009, and we're all given the "look what you can do without coding!" demos, and the BDC is still 14 pages of XML without proper tooling, and pieces of the framework are "secretly known" to not work at launch (like the PRIME API), and SharePoint still produces noncompliant HTML out of the box, and I'm still handwriting Feature manifests, and I'm still handwriting CAML, and I'm staring at a doubly-XML-encoded internal field name passed as an argument to a custom XSL function emitting HTML and all this is wrapped in a XML-encoded WebPart tag, and a new SPDataGridView is released and it adds 50 more undocumented methods and properties, and we need to rewrite all our SharePoint 2007 customizations just to make them work with v14…well, we'll see. If MSDN rolls out a SharePoint 14 resource center and it has 3 articles and 50000 useless Sandcastle-generated stub pages—gold rush or no—it may be time to bail.

So what are you going to focus on, Peter?

I have three words for you.

FUN.
DUH.
MENTALS!

Project management (not getting a PMI cert; instead, the meat of project management). Proper object-oriented design. Programming languages (plural). HTML/CSS/JavaScript (they're not going anywhere).

Personal projects!

Programming for fun—something that may even inspire me!

And hey, maybe SharePoint 14 will be a pleasure to work with; maybe I'll jump on that bandwagon…I'm the guy who owns sharepoint14.com, after all.

Monday, September 22, 2008 8:00:22 AM UTC 

Estimating is difficult to begin with, but add on

• a new technology stack with which you're unfamiliar,
• sometimes unreliable and often undocumented API (I can back this up with examples, just take my word for it),
• tight integration/reliance on Active Directory, SQL Server, IIS, Exchange, Office, and Internet Explorer, all of which may fail in difficult-to-debug ways;

Given all this, how much risk are you taking on if you provide a tight estimate? If you haven't tackled three or more Visual Studio Workflow projects, for example, how do you even go about getting a ballpark estimate on your first one?

On a recent project I put a 10x difference between my lowest and highest estimate for an SAP integration task, and everyone complained to me about how "there's too much variance." Well, what should I put there, I've never tried it before!

This isn't a passive-aggressive way of trying to win a work argument via my blog. If I start doing that, dude, let me know, I don't want to be that guy.

But it is a cry for help. How are we supposed to estimate SharePoint development tasks? If you're not an experienced SharePoint team, and by "experienced" I mean experienced in building the exact same solution, using the exact same method calls on the exact same SharePoint objects—if you're not truly experienced, then how do you even attempt estimating?

When you may be blocked anywhere between zero to ten times on a task, and each blockage may take up to a full week (or more) to resolve?

When your project approach hits a dead end? A brick wall?

In traditional build-it-from-the-ground-up systems, you can estimate the complexity of your requirements and envision a rough idea of what your solution will look like, and compare the complexity of your new task with similar tasks you've done in the past. This works for traditional development, so long as you double your estimate and increase it by an order of magnitude :). The idea is, when you're building it from the ground up, estimating is "as simple as*" estimating your effort based on the complexity of the problem.
* saying estimating is "as simple as" anything is an implied joke; you may laugh now

But if your task says "write a custom field that pulls data from SAP," or if someone says "hey, let's build this web part with Silverlight!" …how do you put down an estimate? When the task, performed by an experienced expert (see above for my definition of experienced) takes an hour, and in reality the bulk of your time is spent discovering how to do the one hour task?

What do you do when your honest estimate is, "this will take between 40 and 400 hours?"

I don't have the answer

I'm just throwing this out there because no one even talks about it. So here it is. When you're on a project and someone asks you for an estimate, and you answer "4 hours to 4000 man-months", and they laugh, hey—you're not alone. I can't figure it out either.

UPDATED 2008-10-21: I made mostly grammar fixes—so if you notice some changes, no, you're not going subtly crazy, this post changed a little.

Monday, September 22, 2008 8:00:22 AM UTC 

A while ago I came across a paper written by Niklaus Wirth. Well, okay, I don't read "papers", but I do read programming.reddit.com, and so came across some guy who does read papers, whose story made it to the reddit front page. I'll just link to him.

So here's how this goes: I'm only going to quote a small section of the original paper, then ask, "does this sound familiar to you in any way?"

Okay, here goes. Niklaus Wirth:

By contrast, modern languages are constantly growing. Their size and complexity is simply beyond anything that might serve as a logical foundation. In fact, they elude human grasp. Manuals have reached dimensions that effectively discourage any search for enlightenment. As a consequence, programming is not learnt from rules and logical derivations, but rather by trial and error. The glory of interactivity helps.

…so, ring any bells?

Wednesday, September 17, 2008 8:00:30 AM UTC 

I've been fascinated with learning as much as I can about SharePoint 14, also known as "the next version of SharePoint." You can see the list of everything I can find publicly here: http://www.pseale.com/blog/SharePoint14EverythingWeKnow.aspx (nothing much has changed since I wrote that post).

One of the advantages of picking up these little tidbits (as well as reading every SharePoint blog known to mankind) is that you can make better guesses.

So, what follows are my educated guesses as to what technologies we'll be adding to the SharePoint arena, with the idea that you can brush up on some of them in preparation. Or just get depressed. I'll help speed the process, either way.

Technologies we'll see in v14

PowerShell

I've seen a hint dropped (by a MS employee) that PowerShell will be part of the next SharePoint framework. I'm guessing admin-style cmdlets, maybe better wrappers than using stsadm. I've become accustomed to using stsadm, but could definitely use some PowerShell shortcuts, especially if they could balloon the number of things PowerShell exposes. There's a lot of potential for using PowerShell as an admin interface, way more than even Exchange 2007 is realizing. And all this on top of everything PowerShell gives you today.

Anyway, learn PowerShell! It's going to be the easiest thing to learn, by far, and as I tell everyone, PowerShell is one of the few technologies that has saved me time, even including the time spent to learn it.

Windows Communication Foundation - WCF

It's inevitable that SharePoint will expose WCF services, and even if SharePoint doesn't, then we'll still eventually be writing our services in WCF (as opposed to ASMX web methods). So, learn the ABC's of WCF. Sahil Malik covers WCF for SharePoint 2007 on his blog, so maybe that's a good place for you to start.

We can hope for WCF services that replace the SharePoint built-in web services of today, though I honestly don't know how they'd improve. Maybe they'd be able to generate list-specific services? I.e. instead of a big "GetListItems" that takes the equivalent of a SQL SELECT query, they'll have something more context-specific? We can hope.

Silverlight 2.0

I'm not sure what form this will take; at minimum we'll have Silverlight Web Part templates. We can also maybe get some "all-Silverlight" site templates, where you visit the site and you don't see a single bit of HTML. As far as I'm concerned, Silverlight looks like a more difficult way to do the same thing we're already doing, but hey, that's me, I'm cranky. I'm going to skip this whole Silverlight thing until someone proves that it's easier to work with Silverlight than the equivalent solution using HTML/CSS/JavaScript; I'll re-evaluate periodically. I'm way too overwhelmed with learning to take Silverlight seriously, so don't expect me to jump on this bandwagon anytime soon.

It's also possible InfoPath will use Silverlight; think about the possibilities (but as always, don't get carried away :) ).

Also, if you're learning Silverlight, then you'll be learning WCF, gotta talk to the server from your Silverlight client somehow.

Claims-based authentication

This is a big shift and, while it may not happen for SharePoint 14, it is probably worth learning about anyway. Here's the NetworkWorld article from 2007 discussing the possibility.

.NET 3.5, IIS 7, Windows Server 2008

I'm including all these for completeness' sake; duh.

Stuff we haven't heard about yet

This section is also included for completeness. While I don't think v14 will be as big of a shift as 2003->2007 was, what do I know? Maybe the entire Dynamics platform will mash itself inside SharePoint; you can only hope (dread) the possibilities. I'm just saying here that no, I don't know what else we're going to get. I can't imagine what Access Web Access will look like. I don't know how Groove is going to work in the new version (are we going to get offline support out-of-the-box? I don't know). I don't know what the SQL-table-backed-lists will do and how they'll work. I don't know if they'll revamp the BDC. There's lots I don't know.

New slices in the SharePoint pie

Master Data Management (MDM)

This will be big. Read up on MDM. You're not going to be an MDM expert, but at least you'll know that it's not as easy as running UNION queries on disparate data sources. Hopefully that doesn't go into the marketing packet.

FAST search (Enterprise Search revamp)

This will change SharePoint's search engine, maybe not in time for SharePoint 14, but maybe so. Microsoft has proven they are able to do out-of-band improvements on their Search platform (MS Search Server), so even if this doesn't happen in time for RTM, you won't have to wait until v15.

Here's a discussion of the FAST Search acquisition on CMSWatch; you can read elsewhere (MS Enterprise Search team blog) about the acquisition, and of course you can check out FAST's product itself, as it exists in its non-integrated form.

Do you have time to learn all this?

If there's an argument to be made for specialization inside of SharePoint, combining this with the existing SharePoint 2007 architect skillset should do it. Don't even try to stay on top of all this; you can't.

Wednesday, September 10, 2008 2:45:24 AM UTC 

In my learning journey from earlier this week, I forgot to mention something:

Don't dispose SPWeb or SPSite objects you get from the "properties.Feature.Parent" property. This rule doesn't come from me—I got this information from a reliable source.

So, here's a little visual guide:

 

From my brief survey of CodePlex projects, it looks like most people were following this rule anyway.

Tuesday, September 09, 2008 11:02:20 AM UTC 

Read about Patrick Tisseghem, sad

Tuesday, September 09, 2008 10:29:40 AM UTC 

Update: 2009-03-31: Go visit someone else's blog

Go visit Keith's blog for the definitive SharePoint disposal article. Also be aware that Microsoft has released the SPDisposeCheck tool.

That's pretty much it.

Introduction

If any of you out there are looking to find reasons why SharePoint is a "challenging" developer environment, you need look no further than explore the guidance on when to dispose SPWeb and SPSite objects. I was reading Chris O'Brien's blog (which is a veritable font of knowledge on SP dev topics) and found this page, discussing his generic approach to disposing objects, and figured I'd better post this so I can just get it out. Keep in mind folks, that every single SharePoint developer in the world must deal with this issue.

To introduce the problem: Your friendly SharePoint objects SPList and SPWeb will on occasion interact with a COM component. Each pop, for each reference, they hold between 1MB and 2MB of memory. That adds up quick.

So far we're okay, because .NET has a facility for disposing these unmanaged resources (among other things)—through the IDisposable interface. This is why we tell everyone to wrap their code in using blocks—it's to make sure that the unmanaged resources (like the COM components I mention above) are properly released. If you're writing code in SharePoint and you're don't know what a using block is, go read up on it. If you're ignorant of this, it isn't the end of the world for you, but it's close.

Where were we again? Oh yes—SPLists and SPWebs hold unmanaged resources and must be disposed. The "duh" solution is of course, to wrap everything in a using block.

Unfortunately, disposing in SharePoint ain't easy

The first problem you'll run into is when you try and use SPContext.Current.Web - a static getter method. Do you dispose the SPWeb object here, or not? Note that if you dispose an object that's in use elsewhere, you'll cause new problems! Problems you didn't forsee!

This is starting to feel like whack-a-mole. Whack the unmanaged resource mole, up pops the "you whacked the wrong mole" mole.

Whack the "you whacked the wrong mole" mole

So now we know, okay, dispose SPWeb and SPSite objects, but don't dispose the SPContext.Current.Web object. Okay, two things to remember, I can handle that. Unfortunately, we're not done—we're not even close to done. We still have to deal with the "SPList.Web" reference mole, "SPSite.RootWeb" reference mole, the "enumerating through a collection" mole, the "I didn't use the unmanaged resources, this time, and thus don't really need to dispose" mole, and so on.

Disposal strategy Whack-a-mole

There are a bunch of resources you can read on this topic, including:

• Best Practices: Using Disposable Windows SharePoint Services Objects [on MSDN] - good introduction to the topic, but not comprehensive - thanks Jeremy for the link
• Roger Lamb's [MS employee's] nearly-comprehensive list of dispose patterns by example
• Disposing SharePoint objects - what they don't tell you by Chris O'Brien [MVP], as linked above
• Keith Dahlby has posted several times about this:
  • http://solutionizing.net/2008/07/23/spsite-spweb-leaks-revisited/
  • http://solutionizing.net/2008/08/10/is-splistparentweb-a-leak/
• …probably lots of others I haven't referenced

I'm still playing whack-a-mole, but I'm holding steady at the "apathetic about which mole to whack because there are too many moles to keep track of" mole. Oops, sorry, "apathetic about which mole to whack because there are too many moles of which to keep track—there, now I'm not ending my sentence with a preposition" mole.

I'm probably stretching the whack-a-mole reference too far

If I've lost you in the craziness above, let me just summarize by saying: a) object disposal in SharePoint isn't simple, b) I've given up looking for the "perfect solution".

Let's tackle my less-than-perfect solution for object disposal next.

Peter's (newer—thanks Keith) imperfect strategy

Let me first say that I wrote up "my current technique" in this post and secretly published it to my blog. Unfortunately (or fortunately) "secretly publish" doesn't work as well as it should and I got comments. My original approach, which was my firm stance as of Sunday, has changed quite a bit, it's been a full 24 hours. I've taken a little learning journey in the comments below, and in a post on the MSDN forums, where I got an authoritative answer within an hour. Nice!

So, anyway. My newer strategy is to basically copy either Chris O'Brien's or Keith Dahlby's approach, with the one extra little tidbit thrown in: if I'm unsure whether or not to dispose an SPWeb object, I will gather the minimal information necessary from this SPWeb object (e.g. RelativeUrl, Id property), and spawn a second SPWeb object, which I am safe to dispose.

Here's the full algorithm that I use to guarantee I'm safe:

• Follow published guidance to figure out whether the method call/property getter I'm using will produce a safely-disposable SPWeb or not. This applies really to any IDisposable SharePoint object, including SPSite, PublishingWeb(?), Area(?), and objects of which I've never heard. For this example, SPWeb.
  • If definitely yes,
    • Great! Enclose in a using block.
  • If we're unsure for any reason,
    • Use the SPWeb object we're given, and somehow, with the lightest footprint possible, create ourselves a second SPWeb object, that we're guaranteed we can dispose!
    • We can use this "spWeb2" object and proceed along our merry way, disposing it with glee!
  • If definitely no,
    • Great! Don't dispose it!

Tear it up

Feel free to tear this approach up (assuming I or someone else hasn't already done so). I can take it.

Tuesday, September 09, 2008 8:00:17 AM UTC 

I recently read Matt Blodgett's post called SharePoint is the Future of Microsoft-centric Application Development, which is interesting because there's a grain of truth in what he says (not more than a grain though :) ). I want to focus today on the following quote:

[…referring to SharePoint projects] Clients are amazed when they see 80% of their app done in a week. What they don't see until later is the weeks/months of excruciating work it then takes us to crowbar the remaining 20% into an ill suited platform.

Ok. I'm only quoting Matt above because his post was the closest at hand when I need to quote someone—I hear this everywhere (and have probably said it myself several times before). It's completely correct; you want to try and solve the customer's core problem, something SharePoint's out-of-the-box features are good at doing quickly.

But, I need to elaborate. Something I'm not sure everyone is getting, is the corollary:

Lesson: You can stop after the easy part!

You can stop at 80%! This sounds obvious to me, but maybe someone out there hasn't had their eureka moment. "Now wait a minute," I hear you telling me, "we need all of this to work, not just the easy parts!" Okay, let me tell you a story.

Story time!

Earlier this year, I was asked to estimate how long it would take to build an employee vacation scheduling solution in SharePoint. As I dug into the problem, the complexity jumped out at me. We were going to pull some data from the ERP system, run a gauntlet of vacation prioritization, and even include a period of "open enrollment" wherein employees' requests would be put in a prioritized queue. And write something back to the ERP system. And supply complex reports. And so on; it was clearly not something we'd be able to solve with out-of-the-box features and would take a while.

After asking the client about cutting bits from the app, they explained that they couldn't cut features; we'd have to do it all.

So, we had a relatively complex app to build, and limited leeway. What were we to do?

In the end, the client and we agreed on not implementing this as a SharePoint solution. We parted ways, so to speak.

Now, you may ask yourself, where was the "80%, then stop" in this story? I did say, after all, that they didn't pick SharePoint for the app, right? Right.

What I didn't tell you is that SharePoint was already running their vacation scheduling; it was doing the 80% of what they needed, and they came to us asking for the remaining 20%. Because SharePoint was not the most efficient way to solve this complex problem, and because there were no outside extra benefits to using SharePoint, there was no point in implementing the solution in SharePoint.

Lesson learned: wait, what was the lesson again?

Ok, here goes:

Lesson: You don't have to use SharePoint!

You can develop apps outside of SharePoint! I'm sorry I typed this up; it's too obvious. Let's take a more helpful approach, something you can use:

Lesson: Know 'why' you're choosing SharePoint as your platform.

Even if you need to run your app in SharePoint, so to speak, what specifically do you need to run in SharePoint itself? Do you need to store your data in custom lists? Do you need to run all on the same physical web server? …now are you sure about that?

I was inspired when I saw kroger.com, which is a public SharePoint site by the way, and how it handles integration—while the look-and-feel is all the same, it's clear that they're only using SharePoint for its publishing features, and "going outside" for any of their complex needs. They don't even try to get everything onto the www.kroger.com domain; if you look carefully you'll see you're being redirected to different servers. Which is fine. If simply "allowing us to run multiple domains" cuts costs by 80%…

Personal bias

I have not worked on any sort of ECM or huge portal project. If I had, I might have had a different opinion; in those cases, you may "need" to have everything run in SharePoint; maybe the data's stored in SharePoint document libraries, or maybe you need need everything to run natively on your portal. But as I mentioned above—if you don't know why you need it to run in SharePoint—then maybe you don't need it at all!

Sunday, September 07, 2008 4:44:06 AM UTC 

In a previous post, called Thinking Creatively, I asserted that in SharePoint, "we are forced to think creatively" in crafting solutions. Like when someone asks you to create a custom web part on the intranet home page to do employee search in SharePoint, you are supposed to say "hey, aren't you really asking for People Search?" And the idea is that you would then work with your customer to find the real solution to their problem, not just blindly follow directions. Maybe People Search works for them, maybe you end up writing a custom web part or configuring SharePoint's search. You solve the real problem.

Well, I was wrong.

If you remember to do so, you'll think "out of the box" in  "dynamic way" and "harness the synergies" et al, as mentioned in the scenario above. But if you forget, if you slip up, guess what?

Or what happens if your client is belligerent and wants it done their way? FYI for everyone who knows me, I'm not saying this is my client :), but have instead heard secondhand about these situations.

Bad times.

In Summary

Remember to argue with your customer! This is a fundamental aspect of any project, whether it's SharePoint or not—it just happens to sting more (a lot more) in SharePoint when you're forced to follow strict requirements.

I'm not saying you should add a "fistfight" bullet point to your next customer meeting agenda—or maybe that's not such a bad idea after all! Treat your meeting like a boxing match: instead of doing bullet points, say "ROUND 1: do we really need to brand SharePoint?" and "ROUND 2: Can we change the way this app "runs in SharePoint"?" Set the Outlook meeting location to be "THE OCTAGON". Bring a metal folding chair and lean it next to the door, in case someone needs some extra "chair therapy" across the back. Go wild, it's your meeting.

Thursday, August 21, 2008 8:00:39 AM UTC 

Announcing: Houston Techfest 2008 – 09/13/2008 at U of H

The second annual Houston Techfest takes place on Saturday, September 13^th, at the U of H main campus. This year the Techfest has expanded to over fifty sessions. Make the time to attend!

This is a free, community-driven event that (this year anyway) features mostly Java-related or .NET-related content, with a smattering of just about everything else.

This is your only chance to attend such a wide variety of in-depth sessions, in one place, without getting on a plane and flying somewhere. If you’re bored by flashy tech demos, attend the Methodology, Security, and ALT.NET (AKA Continuous Improvement) tracks. On the other hand, if you love flashy tech demos, we’ve got those too! Lots of them! I'm also happy to say that most of the sessions cover advanced topics, so if you're an information nut like me, you'll find something new at the Techfest. And it’s not all business either; there'll be fun sessions on XNA, Deep Zoom, and Robotics Studio. Don't forget, the price is right!

Houston Techfest 2008 Topics List

==> To attend, you must register! <==

1. Register as a user on the Techfest site. Here is a direct link to the registration page.
2. Without closing the browser (and without losing your session) visit the Agenda page and select each session you'd like to attend.

Links

• Houston Techfest home page
• Agenda (calendar-style view)
• All sessions, including descriptions
• Prizes (yes, there will be prizes)
• Football games you will miss by attending the Techfest - it's important to know what you'll be giving up

Finally, Lando

I don't know how, or why, but what I do know that I promised Lando, and so I must deliver. So, without further ado:

Wednesday, August 20, 2008 8:00:12 AM UTC 

First off, I'll point out I'm not an SVN expert, nor am I a CodePlex expert. I've just figured out how to do the basics, and found that the process was way too complicated. It's like you need a diagram to figure out how to make it work! What I've created below is that diagram: here's how to do a checkout of your CodePlex project from SVNBridge!

Step 0: I assume you have installed SVNBridge and a SVN client (e.g. TortoiseSVN)

If not, make that happen first.

Step 1: Figure out where your CodePlex project is hosted

Step 2: Figure out where your SVNBridge proxy is running

(NOTE: do not mess with the SVNBridge "Proxy settings" unless you're the one person in the world still running behind an authenticated HTTP proxy. You're not, so don't mess with it)

Step 3: Now we're ready to CONCATENATE!

Step 4: Let's try this URL in Tortoise!

 

RESULT: Success!

Footnote: Authentication is another matter

If you need to do updates/commits/etc, you'll need to be sure you're authenticated properly. Use a slightly modified version of CodePlex username and password:

SND\cp_[[yourCodePlexusername]]

(yourCodePlexpassword)

Footnote: If you can't see the source code on the CodePlex site, it doesn't exist

If you're having problems accessing the source through SVNBridge, but it's not spitting out any errors, then consider the possibility that the source isn't public yet.

Tuesday, August 19, 2008 8:00:31 AM UTC 

One of my friends is starting his first SharePoint project, and I had a few words of advice for him. He doesn't know exactly what his project entails, I don't know what his project entails, but we will start with the following assumption:

The following advice assumes that we are working on a small-to-medium sized, line-of-business application, possibly something attached to a traditional portal (i.e. intranet). We are not working on a BI project, not rolling out an entire enterprise intranet, not creating a publishing site (WCM), not even doing formal records management.

First tip: Read this other guy's post on the subject

I found an excellent post by Mark Jones written mid-2007, which is ancient history in SharePoint terms. If you can't be bothered to read both his post and mine, choose his: Mark Jones on SharePoint projects (original post disappeared, this is a link to the post in my Google Reader)

Second tip: Use either WSPBuilder or STSDEV in Visual Studio

It is well known that no one really uses the Microsoft-created Visual Studio Extensions for Windows SharePoint Services (VSeWSS). Well, no, if everyone already knew, then I wouldn't be writing here, would I now?

So let's start with a fresh mind. You're new to SharePoint, and you need to do … something, let's say change the PDF icon that appears in document libraries on your SharePoint farm. Terrible example, I know!

I'll assume you have rudimentary knowledge of SharePoint's Solutions and SharePoint's Features. If not, read up on it, there are intro books and articles out the wazoo. I'm not going to into depth here; read up on Solutions and Features elsewhere.

Moving along.

So you fire up Visual Studio. You've already installed VSeWSS, and now you've got this "SharePoint" tab in Visual Studio, and you click on it.

I'm not here to only throw rocks; I'm here to say that, unless you want to endure a great deal of pain, don't stick with only VSeWSS.

You have alternatives:

• WSPBuilder via the WSPBuilder extensions
• STSDEV
• "Naked" class library project with WSPBuilder as a post-build task
• "Naked" class library project with MAKECAB (!!!) as a post-build task/MSBuild task

To the initiate, to the untrained eye, these are all equally good options. They're not. Let's work through the list again, this time with a critical eye:

VSeWSS - Visual Studio Extensions for Windows SharePoint Services 1.2

You're going to have to install this as it is the easiest way to get the SharePoint-specific Workflow activities. Otherwise, don't use it. Search for others' complaints about VSeWSS; general consensus is that it's okay, but painful.

STSDEV - a proof-of-concept utility

Link to STSDev project home 

Contrary to the aggravating unnecessary disclaimer prominently displayed on the CodePlex project home page, STSDEV can be useful in actual, real-life projects! Shh, don't tell anyone, they might figure it out!

Aside from my nitpicking, this is a good tool.

• Generates the Solution manifest.xml for you, for limited scenarios
  • In fact, generates an entire project skeleton. This includes the feature.xml manifest, something for which you'll be thankful if you're learning the ropes.
• Builds your Feature event receivers for you
• Screencasts on how to use it! Ted Pattison is an excellent communicator and presents three screencasts, available from the 1.2 (previous release) page on CodePlex.
• In other words, STSDev is a good way to start off a new project, assuming it has a template for the thing you're trying to build. I don't use this as I've pretty much settled with WSPBuilder, but acknowledge it's a good solution, and will definitely use it in the future if I find STSDev has something WSPBuilder lacks.

WSPBuilder extensions

Link to WSPBuilder project home

The WSPBuilder Extensions are relatively new—they add Visual Studio project templates, quick shortcuts, and WSPBuilder-specific item types. LIke the WSPBuilder command-line tool, the WSPBuilder Extensions also completely eliminate the pain of Solution packaging.

• WSPBuilder project template is a real, new Visual Studio project template. It saves you the trouble of manually configuring your Visual Studio project to work with SharePoint; stuff like making a .snk file, adding a "Build WSP/deploy/upgrade/uninstall" option to your project (further note: uses SharePoint object model directly for faster performance); and other little bonuses (see screenshot above).
• Downside: Unlike STSDev, you're locked into WSPBuilder Extensions once you start using the Project Template. I'm not worried, because I would feel comfortable switching the project over to a raw class library, but you may be allergic to dependencies like this.
• New Item templates (like "Web Service") - I haven't tried these, nor have I heard reports of how well they work; so I'll just say—try them out, if they work for you, great; if not, definitely don't use them.
• Assuming you arrange your artifacts in the correct directory structure, WSPBuilder will eliminate the pain of building Solutions. See my review of WSPBuidler.exe below for more details on what this does.

WSPBuilder.exe - command-line tool only

Link to WSPBuilder project home

In ye olde days, as far back as almost a full year ago, we were not blessed with either STSDev or WSPBuilder Extensions. We made do with what we had; WSPBuilder.exe.

WSPBuilder.exe is great because it is focused: it does one thing, and it does it very well. What is that one thing, you may ask?

It eliminates the pain of Solution packaging. That's all it does. To be honest, I don't even know how to build a Solution manifest XML file, and I attribute my brazen, unashamed ignorance to WSPBuilder. I don't need to know because WSPBuilder takes care of that for me.

The pain begins, however, with the rest of your project. WSPBuilder Extensions address some of this pain, and I recommend you start with WSPBuilder Extensions for new projects, but you'll find many projects that were started during the "WSPBuilder-only" era. Don't make fun of these old projects; we can only hope someday that we will be able to look back and say "what primitive tooling we had in 2008!" Just like you're wont to do when you see a project that uses WSPBuilder.exe or MAKECAB.exe to build its Solutions. We can only hope the tooling improves.

If you want to see a simple example of how to work WSPBuilder.exe into your Visual studio project, I demonstrate one way of doing this in my terrible CodePlex project. F5 is my build, and I'm (almost) unashamed!

Raw MAKECAB.EXE

This seems to be the preferred method for the "old school" SharePoint experts who have been around. I assume that, for them, they have learned how to use MAKECAB.exe and build their own Solution manifests. Ugh. Ultimately if you know what you're doing, this is as good as any other way, but for the first-time SharePoint developer, this isn't a good option. I want to point out MAKECAB's existence because the MSDN-sponsored articles and project templates all rely on MAKECAB.EXE. You don't have to listen to those articles; save yourself some pain and pick something else. There are far better alternatives to hand-crafting your own Diamond Directive Files and hand-crafting your own Solution manifests. Look around; better ways of doing this exist!

Recap

Both WSPBuilder Extensionss and STSDev are good solutions for the budding SharePoint developer; they both provide some tools to help you create and build SharePoint projects in Visual Studio. They are in no way complete tooling; you will still experience pain, but they both help greatly. Try both out, choose one and go.

Just don't stick with only VSeWSS or only MAKECAB; save yourself some pain and choose one of the alternatives.

MORE TO COME!

This post has gotten ridiculously long, and I barely just compared the major competing developer tools! We're nowhere close to done; more posts to follow on this subject.

Monday, August 18, 2008 8:00:15 AM UTC 

When converting an existing "please leave us a comment" form, I had a eureka moment: SharePoint's user profiles are incredibly useful. Let's do this graphically:

Before - ask the user, again and again

After - SharePoint stores user data automatically in the "Created" field

Well, you get the idea anyway

The point of this exercise is to show the power of integration. Specifically, user data like phone, email, work location, department, etc. In an ideal world, SharePoint will pull this data for you from wherever it's stored. HR type data from the HR/payroll system, email from Exchange/AD, and so on. With proper integration you can remove all the pain of online forms; you can completely remove the five "who are you again? and what do you do again? and how do I get a hold of you again?" fields. They're gone; thanks to proper system integration, you already know and don't need to ask these questions again.

Footnote: it's not all gravy

I must acknowledge that there is no easy way to grab and show all this extra data from the user profile without writing code. I wish SharePoint would have made this easier, but alas, 'twas not to be. In the browser, you can certainly click through to see their user profile data, which is good enough for the example above, but that won't always suffice.

Second footnote: user profiles in all their glory require MOSS Standard or better; this is not a WSS-only feature.

Saturday, July 19, 2008 4:30:43 PM UTC 

This will be quick.

QUESTION: is there anyplace in, near, around, or even remotely near downtown Houston, where I can go get free wifi? Where they're not trying to shoo me out the door as soon as I walk in? The first person to say "Starbucks" gets shot; they don't give out free internet anymore, since 2000, or way long ago. Also in the news, they're closing a bunch of stores. ARE THESE TWO THINGS RELATED? You make the call!

Back to business.

I'm looking for something as awesome as a Panera. 1) Free wifi, 2) Encouraged/allowed to stay for 2+ hours, 3) Fountain drinks are a huge bonus, but not required.

And before you ask, no, for some reason there are no Panera's near me. Anywhere near me. Allow me to illustrate!

Sunday, July 13, 2008 4:50:49 AM UTC 

Ok, so I'm burned out.

I realized this while digging through my queue of SharePoint-related blog posts. I wasn't reading all the technical bits (which is normal), but I realized I didn't even want to know what any of the blog posts were about. This antipathy is new.

But I'm not writing here, today, just to tell you I'm burned out. I'm going to hit a few topics and hopefully draw them all together at the end. Maybe by the end we'll figure out why I'm so burned out. Let's do this.

Do you have time to learn everything?

I came across a post (via DotNetKicks) asking if we have enough time to learn everything in the Microsoft stack. It's an interesting question, and always leads to good conversation. "How do you keep up?" is a fun question to ask.

Unfortunately the author, in what has to be a misguided set of priorities, has chosen to devote all his learning efforts to LINQ to SQL and the Entity Framework. Which is a ridiculously bad idea. Don't worry, I told him the same thing in the comments, I'm not just sniping from afar.

What bugs me about this idea is that it's almost the exact opposite of my learning priorities. LINQ to SQL and the Entity Framework are almost dead last on my list—how did he end up choosing these two volatile technologies over everything else?

Think about that for a second. How did he arrive at the misguided conclusion that learning these two things was a good idea?

If you're interested in my specific critiques, they're all in my comment on his page; I won't repost them here.

When doing research, always consider the source

I also came across a discussion started by Andrew Connell on how CMSWatch is biased against SharePoint.

If you're unaware, CMSWatch is a site that sells a $1450 SharePoint report that critiques SharePoint in a variety of categories, as well as provides advice. Andrew Connell is an MVP and is a known authority on SharePoint WCM and developer topics.

AC's point was that not only is CMSWatch biased, but they've also caused him extra trouble in the past—he explained that he had to 'clean up' after them—where he has had to expend extra effort convincing clients that SharePoint can properly do WCM. People who haven't been 'poisoned' by CMSWatch don't require this extra effort.

My comment to him (poorly worded and not in total disagreement) is that I'd rather have CMSWatch and their criticisms of SharePoint than not—I'd rather "clean up" by convincing clients that SharePoint can work, rather than "clean up" by convincing clients that SharePoint isn't the 'everything solution.'

I'm not here to weigh in on the actual discussion of whether CMSWatch is biased or not; I don't know, I haven't seen the report. I will say that this summary written by one of the principals is unfair (go see the 'Ugly' section), but for the most part I'll stay out of it. I don't know.

Instead, the question is: why do I encounter SharePoint overexuberance more often than SharePoint skepticism?

Oren Eine at the Future of .NET Panel on DotNetRocks

A while ago I listened to a podcast, probably my favorite DotNetRocks episode of all time: DotNetRocks Show #346: The Future of .NET Panel. Even if you're totally bored by me, go check out the site and download the podcast, it's a fascinating (and you'll be happy to hear, civil :) ) discussion of .NET development in general.

The thing that really got me going is at ~1:06:00 into the podcast, when Oren Eine describes a challenged deployment of MS CRM customizations:

Question here: why did Oren's story strike such a chord with me? He wasn't talking about SharePoint at all!

Do Less. Get More. Develop On SharePoint. - www.mssharepointdeveloper.com

Also launched sometime recently is the SharePoint Developer introduction for .NET developers. Instead of writing, I'll express myself by defacing a screenshot of the front page of the site:

The question for you, dear reader, is why am I so virulently opposed to including Silverlight in the list above? It's the future, right?

And what's the deal with the Entity Framework vote of no confidence?

This is way too big a topic to cover properly; I'll just assume you know what I'm talking about and skip straight to my leading question.

Why does anyone care if the Entity Framework is released as-is or not? What difference does it make to them; they can just use something else to do data access! Why can't they just live and let live?

"SharePoint is a guild", by Tony Byrne

I also recently attended a presentation by Tony Byrne (from CMSWatch, yes, see section above). I've covered the session in more detail here, if you're interested; the quick summary is that he was providing a quick critical review of SharePoint, a sort of summary of their report. One statement that caught my attention was that he likened SharePoint to a guild. I don't recommend you take this metaphor too far, I'm not a medieval scholar and I don't think he is either—but the idea is that there are a select chosen few who are in the "guild" and have collected secret, arcane knowledge. Those outside the "guild" are not privy to the same knowledge and benefits of those in the "guild".

We could have a very long and unproductive conversation about this, there's several ways to go with it. But why does this crazy idea of a SharePoint "guild" have any merit at all?

Ok, let's pull all this together

We've wandered far and wide today. We've talked about some random guy's technical learning queue; we've discussed the impartiality of CMSWatch; we've heard a story of a painful MS CRM deployment; I've defaced a screenshot; we've approached the topic of the Entity Framework before running away; and finally, we've talked about medieval guilds.

Where does all this relate again?

Ok, I'll try to boil this down. [re-reading the section below, it appears that I failed to 'boil it down'. Too bad; we're stuck with it. -ed]

Overexuberance, belief that improvement is inevitable

What I experience on a regular basis is an overexuberance, a faith in the current crop of Microsoft's technology, as well as the common belief that future improvements are inevitable. In reality, however, no technology is perfect, and many improvements perceived as inevitabilities will not materialize.

Some of this overexuberance is the result of aggressive marketing. But I'm starting to realize as well—we're duping ourselves. We don't even need marketing to tell us that, say, Silverlight is going to be 'the best development toolset provided for the web'—no need for marketing, we'll do that all by ourselves!

We believe that SharePoint's WCM featureset will improve, without any evidence or indication from Microsoft that this is the case! Will SharePoint 14's HTML be accessible out of the box? Are you sure about that? Who told you that? No one?

Learning investments with limited time

What seems to be completely ignored by Microsoft is the hidden cost of learning—for each technology or framework or product released, there's an associated cost. Learning is the number one cost of introducing any new technology!

So why is it then, if our attention is so valuable, that when we are presented with the foundations of SharePoint development, we see Silverlight in the list? Not only is Silverlight completely useless to me for SharePoint development today, 2008-07-12 (note I timestamped this assertion), but it's probably actively worse than the competing technologies (which are some combination of InfoPath, HTML, and/or something "not in SharePoint at all")? Microsoft, instead of attempting to ease my learning burden, sneaked in the additional burden of Silverlight…which is useless!

Now hey, I hear what you're saying: so what if they threw a little marketing in there? You can just ignore it, right?  I can. You can. But what about the guy who has heard about this "SharePoint" stuff, and has also seen something about "Silverlight", and hey, look at the website! It says you can make these two things work together! Sounds easy!

Now fast forward a bit. When this poor guy can't deliver, whose fault is it? Sure it's his fault, somewhat. But is it his fault he was distracted for 3 weeks trying to get his Silverlight applet working in a SharePoint farm?

So is this why everyone's so excitable about the Entity Framework? Because it will make their jobs harder in the years to come—not easier?

Now let's hop back to SharePoint's WCM again, but this time, in the context of learning. If you installed the product sometime in 2007 and discovered a) emitted HTML was not accessible, b) branding was difficult, c) content deployment jobs failed often (hopefully you catch this one before go-live!), then were you supposed to think? Sure, if you knew all the right things, you could work around the problem. But what if you didn't? Is it your fault you can't deliver? If Andrew Connell, MVP, can do it, why can't you? If hawaiianair.com (which as I understand does not use MOSS's WCM features) looks so cool, why can't you brand your portal as successfully?

…Oops, sorry, I kind of drifted off while you were talking there, I was reading an article about how to AJAX-enable my web parts. It says here that I can throw in an UpdatePanel in two minutes! What could go wrong?

And the capper. What kind of crazy world do we live in, where someone is encouraged to learn the most volatile and transient technologies over fundamentals! His post had 11 kicks on DotNetKicks; 11 people agreed with him? He's not alone in his opinions?

So I'm burned out

I'm going to go on an "information diet," for as long as I can hold out; I'll be fine, I've been through this before. No technical learning outside of work, outside of my immediate duties. Shut down the Google Reader, close the book, hide the laptop at home. Not a problem, I'll make it back refreshed.

But if I can summarize my ranting above, it is:

Overexuberance in the current crop of technologies, along with faith in unproven, possibly actively unhelpful or worse future frameworks, makes my job harder, today.

 

Microsoft's strategy of releasing frameworks at an overwhelming rate has left every Microsoft developer overburdened with technical learning. Microsoft (and we ourselves!) worsen this problem by pushing new, unproven, possibly useless-out-of-the-gate technologies on ourselves, before they are even ready—and this will make my job increasingly harder for the forseeable future.

Aside: Houston ALT.NET 'geek dinner' at Star Pizza 6PM on Monday, July 14th

If you made it this far, then maybe you'll be interested in attending the ALT.NET informal gathering this Monday, July 14th, at 6PM at Star Pizza.

And as always, the "sweet place to hang out" is on the houstonaltdotnet mailing list; all the cool kids are subscribed.

Monday, June 30, 2008 11:27:04 PM UTC 

Quick link: http://www.codeplex.com/SharePointPdfIcon

 

This is possibly the most important customization you can make to SharePoint. OF ALL TIME.

By default, SharePoint does not include a PDF icon. When you upload a PDF to SharePoint, you see:

 

This is unacceptable!

Manually adding the PDF icon

I'm not here to tell you how to manually add the PDF icon to your SharePoint farm. Steven Van de Craen already does an excellent job, so I'll just link to him.

But manually adding a PDF icon? So 2006-2007. We're in 2008 people, let's get with the times.

Programmatically adding the PDF icon

Recently Steve Goodyear from Microsoft posted step-by-step details of how to programmatically add a PDF icon to your farm. Which is great.

What Steve did not do, however, was go the extra step and provide a working .WSP file.

Ok. At this point you're asking yourself: "Well Peter, since you're not bothering to tell me how to manually add a PDF icon, and you're not going to tell me how to do it programmatically, then why are we here?"

Introducing the most awesome SharePoint Solution package ever

Introducing: the PDF Icon installer!

I've developed a Solution package to change the dreary, dull, blank icon, to a dynamic, Web 2.0, hot hot hot PDF icon!

Feedback

If something is terrible, let me know and I'll fix it. Issues/Discussion page on the CodePlex project page. Or if it's really terrible, email me direct :)

Where to get it

http://www.codeplex.com/SharePointPdfIcon

Saturday, June 28, 2008 11:25:00 PM UTC 

Yesterday and today I attended the SUGDC Summer Regional SharePoint Conference. And by "Regional", we mean Washington, DC and surrounding provinces and territories. Telling the story via graphics:

I'll point out I didn't drive, I flew, though the flying experience probably took longer on the way up than driving would have. Yes, I'm aware that driving can take quite a while; yes, my flight up was nightmarish.

People

As I've been told before, the most important thing about attending a conference is not its sessions; it's the people you meet. I'm not going to call out names, but it was great to be able to meet a bunch of people of differing skillsets and backgrounds. And by differing skillsets, I mean "expert," "expert PLUS," and "Super expert plus, TURBO"—we're all pros here. Of course, of course; me too.

Fun Tidbits in the Sessions

Since I did dutifully take notes, I'll share the fun bits here:

• Errin (whose company is based out of Houston) presented an excellent overview of SharePoint's various capabilities and what projects they're doing. A good throwaway example is the slide that declared: "Documentum: $1.2M annual license. SharePoint: $400K one time fee." …the point being, even if you have to pay to customize, SharePoint is still the cheaper choice.
  ANYWAY, the thing that made this interesting (and different) is that he has done the projects—they're real, they're not all marketing hype. Believe the SharePoint pie chart! Well, kinda anyway, we'll get to that below.
• Errin also mentioned the importance of getting 'quick wins' in selling your SharePoint deployment. Throughout the weekend several others hammered this home as well, and it makes good sense: Step 1: find something that is a good match for SharePoint and is generally easy to do; Step 2: do that thing; Step 3: profit! Funny enough, I hear the same thing spoken about SOA implementations, although the guidance I heard was more like "3-5 projects." Since parts of SharePoint are designed to solve common business problems, you'll find it's, gasp, actually useful for your organization too!
• Interesting: he says the magic # to extract your storage requirements from your raw data is about 3.5. So if you have 100GB of files sitting on a shared drive that you plan to get into SharePoint, then you will need roughly 350GB of storage to accommodate.
• My Site governance: if we get the slide deck, I'll recommend looking at it to find a list of all the things that can go wrong with your My Sites. We're all aware of the big offenses, wait, let me capitalize that: Big Offenses—but we're not sure what else to look out for. Check his slide deck, there's a good summary there.
• Onto the next session! The Red Cross folk presented on their Communities of Practice, AKA their "Neighborhoods", which I'm loosely defining as "small targeted online communities." E.g. for the Red Cross, something many people are interested in is Measles. I don't know why either, something to do with saving lives. Anyway, my take on this whole thing is that the technical problem is solved, and has been since Philip Greenspun's ArsDigita started doing this sort of thing for corporations in the 90s. The only thing that remains is the human factors—getting people to, you know, use the stuff that's out there.
• Morbid tidbit: one of our presenters (I will withhold company) mentioned that one of the great catalysts spurring people to adopt SharePoint for knowledge management was a big pending layoff (RIF). Knowledge transfer was a real, immediate problem for them, and SharePoint provided a good solution. Takeaway: big layoffs drive SharePoint adoption! Slap that on the side of a bus and drive it around at the next conference!
• Hilarious quote: "You want to use a wiki? How much do you use the discussion lists you have on your site? None? Ok."
• As an aside: for the record, you are no longer allowed to use the term "wikis and blogs," as if they belong together. Small wikis are great. Wikipedia is great. Content tied to RSS feeds, which also allows comments, are great. Blogs that you can find on Technorati and can add to your subscriptions in Google Reader are great.  But in none of the 5 situations above, are you combining both "wikis" and "blogs" into one solution. You're not. Furthermore, nowhere are you even combining your intranet-facing content with your public-facing content. "Wikis and blogs" is a meaningless term, and it's growing ever more popular as the Enterprise 2.0 cloud blows in. I will talk about Enterprise 2.0 some other time; the summary is "part duh, part new good things, part money/hype/vendor."
• Also driven home: the fact that SharePoint implementations should not be "hardware + software + configuration + customization" only. It should include a great deal of effort on PEOPLE, PROCESS, and TRAINING. I'm with you there 100%. Training is always a good choice.
• Awesome nugget: when asked how to prevent sensitive or covered-under-regulation documents from being uploaded to SharePoint, Melvin's response was "what do you do for email?" Their inevitable answer is "nothing; we can't control email." The implication then, is that "oh, then why am I suddenly required to control SharePoint?" Melvin claims we can tell people that we will only hold SharePoint to the same standards to which we hold our email systems. I don't know about you, but this is an awesome "organizational hack" and I'm going to try it out.
• Job roundtable: it is clear that a) there's a lot of demand, b) there are at least three distinct roles for SharePoint, c) the "talent shortage" problem is only going to get worse. My advice: train up someone with no SharePoint-specific experience, almost as an intern-type situation. As you may imagine, this was a rowdy session and had lots of crowd feedback.
• Greg Galipeau - attended this intro to SharePoint site definitions development. What's fun is sitting next to a total newbie to SharePoint who is asking, "now what do I do to start a SharePoint project? Which project do I pick?" What was not fun was to realize that, while I was sleeping, WSPBuilder (my tool of choice) went out and added a bunch of features of which I'm unaware. So I'm out of touch. Also out of touch with: STSDEV. Also out of touch with: VSeWSS v1.2 specifically.
• Tony Byrne of CMSWatch, presenting portions of the CMSWatch report as his presentation. I want to talk about this one for a long time. I just want to focus on three things in his presentation:
• One: take a look at what you need, before deciding on SharePoint. I know a lot of us back into SharePoint and then look around, after having purchased the product, and say, "okay, let's try implementing <feature A>! I read it works great!" If you get nothing else from his presentation/report, it's that SharePoint is not equally good at everything. His quote was "it can do anything, more or less."
• Two: The other thing that's unique is that he focuses on the actual product we have today. This is where I think a big disconnect exists: even outside of marketing influence, people believe SharePoint will somehow get better, and assume that improvement is 'inevitable'. If you take away this second thing, just think for a second: what do we have, now, today? Not what we believe the product will look like in the future—what do we have now. Biggest example for me: Don't look at Silverlight 2.0 seriously, yet.
• Third: He claims that "SharePoint is a guild". Which is offensive, but, when you think about it long enough, yeah, it kinda fits. One quick rebuttal: if SharePoint's a guild (which I'm admitting), then it's the easiest guild to enter of all the enterprise systems. If you want to take something positive out of this, let it be: we still have a long way to go before picking up SharePoint is 'easy'. I could talk a long time about this, but will cut short.
• I want to do a separate post on his presentation. Summary: even if you're offended at times, he offers perspective, based on real research/investigation—which is something unique.
• Day 2: Sahil Malik presented on .NET 3.5 topics. I think I have an aversion to developing advanced customizations in SharePoint. This includes basically everything he presented, including hosting WCF in SharePoint 2007, using .NET 3.5 in SharePoint (requires some server config to enable .NET 3.5), and AJAX (Atlas). If I can see it working, even in a demo, I start to believe it can work; until then I'm skeptical. So, anyway, I now (as of ~11AM this morning) believe WCF can work in SharePoint 2007.
• Also interesting, Sahil points out you should use a single "big honkin'" (that's technical jargon for large) physical SQL Server, in conjunction with dev VMs. The more obvious/default choice would be to host your SQL Server instance on each VM; his setup combines all the SQL instances. He says this is best for performance reasons, which, now that I think about it, works.
• SharePoint for small businesses - actually I'm just writing this bullet point to apologize: I came in halfway through and zoned out for the rest. I blame my stomach, I was starving.
• Random quote: "if you don't have policies, you're up a crik."
• Implementing a PMO (and using SharePoint)
• This was an interesting talk because it focused more on the human factors than SharePoint. Something that should be obvious is that if you don't have the process down, installing MOSS or MOPS will not fix your process for you.
• Quote: "…PMO is not a project site [in SharePoint], it's not just rolling up all your project sites."
• Tip: access disparate data, don't recreate it
• Tip: "if I can see everything I need in one place, then … (handwriting is slow…general idea of "I'll be effective)"
• "Transparency" - sometimes there are fights because people don't want to be transparent. Ooh, I feel that one.
• Speaker prefers Lean, but wisely didn't use the word "Lean" anywhere in his presentation. :) He just said "The Toyota model"
• Fun quote for my PMI coworkers: "PMBOK is starting to get stale." This is fun because I can imagine their response.

I just read that, that was a lot of text

Wow, I need to learn to edit, bad me. Well, no way I'm going back and attempting to edit that behemoth; if you made it this far, then there must have been something good in there to keep you reading. Or you're a skimmer.

A+ would buy again

I do generally recommend this conference, especially to those of you in the area, as:

a) It's on a Friday/Saturday, which means you only lose one workday and one weekend day; (strikes a compromise between missing work and hating life)

b) The management track was excellent and had tidbits you can find nowhere else…well, nowhere else besides another conference or a $1400 report;

c) There was no "Intro to Silverlight" session.

Wednesday, June 18, 2008 2:02:02 AM UTC 

…which is very little. I will state for the record, that all this is public knowledge, primarily because I don't have the hookups to get some sweet sweet NDA action. Anyway, onwards.

The following is everything I could find (and ask others to find on my behalf) publicly available about SharePoint. There's not much we know yet.

Quick summary is: beta's coming soonish, MDM is a new slice in the feature pie chart; Enterprise 2.0; FAST search; claims-based auth; bunch of stuff they probably haven't announced. A lot of this has not been specifically promised for SharePoint 14, but may appear in vNextNext, or vNextNever. Who knows, I don't.

Onwards!

Early access - beta soon?

Office 14 TAP: Nominations are open

SharePoint 14 - available internally since February

Surprising/big-impact changes

Master Data Management

SharePoint may use claims-based auth

Thoughts on SharePoint and FAST search

Random SharePoint 14 tidbits

Enterprise 2.0 conference SharePoint tidbits

SharePoint Lists improvements

Ship date in 2009?

Transcript of Bill Gates' SPConference 2008 speech

SPConference 2008 writeup - Microsoft's strategy for vNext

MVP summit 2008 recap

Access Web Access: Speculation

The Bill Gates Interview

OOXML and PDF support

WinSuperSite Office 14 FAQ

64-bit only

The Future of Groove and SharePoint

Rampant speculation

Forrester's educated guesses

SharePoint vNext Rumors

The Next Version of SharePoint

Wednesday, June 11, 2008 4:46:27 AM UTC 

Today's fun challenge on SPOT the TYPO will be a piece of a SharePoint deployment script!

Let's bring on the challenge!

SPOT the TYPO!

(line breaks added for readability; ignore them) 

Still can't find it?

Maybe there's a problem with the image…let's try plain text!

stsadm.exe -o deploysolution -name MySolution.wsp –immediate -allowgacdeployment

Wait, I think I saw something!

Here's the image, enlarged:

I definitely saw something!

Now let me highlight the important bits. Enhance!

 

…enhance!

MYSTERY: why is this dash character wider?

Something is definitely fishy! Let's check out a hex dump and see what it says!

 

Hey, now that doesn't look like the standard ASCII character for a dash! That's not like a dash at all!  In UTF-8 encoding (as this file is encoded), ASCII characters should look like ASCII characters! Right? Right!

Dude, where's my cardash?

It turns out, it's the endash, "&#8211;"—heretofore dubbed "the script ruiner." Read all about it (and I mean all about it) in this incredibly detailed article, about the endash. I'm serious, check it out.

OK, WE GET IT

As is always the case with these "why did my character magically become some other character" mysteries, we can safely blame Word AutoCorrect. When we're typing up technical documentation in Word: blame AutoCorrect. Even when we're typing in Outlook, where we think we're safe: we're not safe, Word AutoCorrect lurks in the shadows! It's hiding under that huge ribbon, waiting to pounce! Ha-ha, those three dots just became an ellipsis! You didn't even notice!

Let's say we're looking at a complex Microsoft Knowledgebase article, oh, say #934838. Let's just say. And let's say that on this article, there are lots and lots of commands just begging to be cut-and-pasted into your favorite text editor, Notepad. Notepad is your favorite text editor because you're not crazy enough to attempt to install another one on the production server. And when you paste this into Notepad, everything looks great on the standard Notepad monospace font. It's a monospace font, think about it.

And then you paste the text directly into the Command prompt, which spits out the generic stsadm error, which is hilarious, because they put the error at the TOP and immediately kick off a 5-page command listing, so you have to dig for the original error message. And yes, the error message says "Command line error." And you're studying, and studying, and you have no idea why this isn't working!

LET ME TELL YOU WHY IT ISN'T WORKING. Because when you zoom in, REALLY zoom in, you'll see that the dash character is exactly two pixels wider! And boy, let me tell you—those two pixels make all the difference! Get out your microscope!

Zoomed-in SharePoint art

If you look at the hyper-zoomed-in picture from above, it almost looks like some sort of awful abstract art. Noticing this, I thought—might as well go with it! Allow me to add a few touches here and there, maybe work with the interplay of light and form and structure, maybe add some tasteful emotion words…and give birth to the worst SharePoint-themed abstract art ever:

As with all true art, each person will take away something different from this piece.

Reader challenge

I dare you, dear reader, to take on the challenge: can you create something worse? You can! You have it in you! Believe in yourself!

Tiny PowerShell footnote

There is a systematic way to find Unicode characters in your strings—just check the integer value of each character. Should be easy, right? I'd say so, yes:

Note I used the "%" shortcut instead of "foreach", and "?" instead of "where". Read it as: convert the string to a char array; for each character, convert it to an int; now filter these integers down to only those with a large (non-ASCII) value. The remaining ints are returned to the prompt, which displays them as best it can. There happens to be only one character today, which is the endash, "the script ruiner."

Anyway, the point is you can guarantee your string is ASCII if you make use of PowerShell's (.NET's) built-in Unicode support and write a simple script.

Tuesday, June 10, 2008 3:54:44 AM UTC 

If you're thinking to yourself "dude, this is wasting my time just reading the headline; how much free time do you have to post about it?" Well, yes. I do waste a lot of time configuring my system, and further waste time thinking about configuring my system, and further further waste time reading others' thoughts when they write about configuring their systems. It's probably unhealthy, but if so it's widespread. Like the flu pandemic.

But I do have a good reason for this tweak.

Eyestrain is a serious problem

Laugh it up; when it's your turn to suffer make sure to let me know and I'll make fun of you.

So anyway, enabling ClearType aids readability and reduces eyestrain.

Plus, it looks cool.

Disclaimer: don't blame me

I've enabled ClearType on the VM running on my desktop. I don't have to worry about bandwidth or getting angry calls from the network folks. Your mileage may vary!

Steps to enable ClearType in Server 2003

• Download the hotfix found at KB 946633. Unfortunately this isn't a direct link; you have to navigate the deadly DRM/registration obstacle course, Takeshi's Castle style. Many that begin the journey don't make it all the way to the end.
  
• Install that hotfix on your Windows Server 2003 machine.
• Reboot as instructed by the hotfix installer.
• Log in locally to the machine and enable ClearType (see these detailed steps if you need them).

BOOM! YOU GOTS CLEARTYPE NOW!

Ok, further steps for RDP

ClearType won't appear over RDP unless you set up your RDP client properly.

• Ensure you have the Terminal Services (RDP) client v6.0. If you don't know, see the screenshot below. You'll need the "font smoothing" option.
• Any time you connect to a server, ensure the "font smoothing" option is checked.

Monday, June 09, 2008 8:00:05 PM UTC 

When trying to convince others why I think PowerShell is hot hot hot, I find myself lacking examples. So, dear reader, I'm writing this list as a sort of "memo to self: don't choke next time and remember one or two of these things." Here goes.

Before the mega-list

If anyone is interested in a particular topic in more detail, let me know. To me, most of what I list below is either a) too broad a topic to cover properly, b) too narrow for anyone else to find it of use, and/or c) just plain obvious.

I can't tell what may be interesting to others; it's all a "solved problem" at this point to me.

Of all these things, since the following is a jumbled, unprioritized, stream-of-consciousness type of list, I'd like to point out that the things I find are most useful are:

a) Object model spelunking - during development, figuring out which function to call, testing out little 5-line scripts to see what happens—little things to help me gain confidence that what I'm writing will work. This is also the most difficult thing to express properly.

b) Visual Studio post-build task - it's just so easy to add and remove bits of script to help me do … whatever it is I need doing. Note this is not necessarily a replacement for NAnt or MSBuild—instead it's something of an informal post-build scratchpad.

c) Deep (focused) administration tasks - like getting the crawl logs programmatically, or getting a list of sites or features into an Excel-friendly format for further analysis. SharePoint allows deep access to its inner workings with APIs that it uses itself—this is unique among enterprisey-type systems.

Memo to self: Powershell + SharePoint

Ways I've used PowerShell + SharePoint:

• Object model spelunking. This includes
• The most commonly used SharePoint objects, e.g. SPSite, SPWeb, SPList, SPListItem. PowerShell's unique (to the .NET space) REPL environment is the best for exploring an object model. You get the full power of an object browser with all the benefits of an immediate window. I don't know how to explain this; you'll just have to try it sometime.
• Neat trick to help you write CAML queries. 1) Make a view that does exactly what you want. 2) Use PowerShell to dig into the view's SchemaXml property. Others have written Windows Forms tools to do the types of things I do in PowerShell.
• Digging into the SPLimitedWebPartManager class.
• Object model manipulations. This usually means really simple things that I do in a dev/test environment, like
• Changing the logo on a site to a different gif
• Changing the default master page property
• Changing lists to Hidden
• Changing fields to Hidden - there was a case where (for whatever reason) the ModerationStatus field became visible; it shouldn't have been; quick PowerShell script confirmed the fix.
• Deleting all items in a list. $site.OpenWeb("/awesomeness").Lists["awesomeness"].Items | % { $_.Delete() }
• Who cares? you may say. What happens if you have 1000 items in the list, that you're programmatically creating every time you run your integration test? Yeah. Yeah, you may need something like this.
• Comparing two fields in a list via PowerShell, looking for corruption in one of them. Amusingly, the problem was that one field belonged to a Content Type, thus was not visible on the list edit page, thus causing us MUCH confusion. Oops! Well, PowerShell was able to help us, just not in the way we thought. By the way, I used PowerShell's metaprogramming facilities to do a 50-property comparison using a one-liner.
• Enterprise Search administration. Includes:
• Getting a copy of the crawl log. Up until a few weeks ago, until the Enterprise Search team released a stsadm tool to do the same, this was quite novel.
• Swapping the "file inclusion list" into a "file exclusion list" in order to aid web crawling.
• Working with wikis - digging into a wiki library and extracting the page titles - it was actually somewhat lame, but hey, it worked.
• Ad-hoc querying of our farm's various sites; who's the owner, when were they last modified, etc.
• Listing the farm's current various peoplepicker settings, to help ease my confusion with their application. This was pre-SP1 before we had the really useful settings available to us.
• Comparing document templates (actual documents) in every library in the farm, comparing pre-migration to post-migration farm. It turned out, during the migration we "lost" two document templates. I'd like to give a big shout-out to SharePoint's WebDAV support, that was definitely a big help :)
• Copying a list using SharePoint's lists.asmx web service. Presumably I was doing some processing on the data, there's not really an excuse for just copying the list from one site to another.
• A different solution: used PowerShell as a "touch-free" solution to generating "reports" on a SharePoint server. I'm not particularly proud of technology choice in this case, but the results were good. This script involved a) pulling data from TFS, b) updating a custom list, c) creating InfoPath form data and "uploading" the forms. It wasn't the best choice.
• Listing all activated Features in the farm (well, actually, close: all sites in a site collection, the site collection itself, the web application, and the farm-level Features). Anyway, listing all activated Features in an easy-to-paste-into-Excel format.
• PowerShell as a Visual Studio post-build action
• Use it to deploy my project
• Use it to do a "fast deploy" - before building, just comment out the sections of the post-build script that we don't need to run.
• gacutil /i for quick code updates to the GAC.
• gacutil /u to ensure that when I run the TestDriven.NET test runner, it is running off of the most-recently-built project build, not off of whatever is sitting in the GAC. With this, I can guarantee nothing is in the GAC.
• Attaching to event receivers.
• Do an app pool reset.
• Warm up the app pool by programmatically visiting a page in the site.
• WANTED BADLY, MANY TIMES: a way to programmatically update a workflow association InfoPath form, WITHOUT having to go through the whole deployment rigmarole. If it's possible at all, I can automate it.
• PowerShell as a "manual setup" for integration tests. We're talking the "copy 100 files to the document library and see what happens" type of integration test, which is … let's just say difficult to achieve without some kind of artifacts. So, in other words, because I was too lazy to set up my integration tests the proper way, I used PowerShell.
• Doing name lookups from the people.asmx web service; checked ~400 names against the people list (and got back a rich list including email address, full name, job title, department, etc) in 5 minutes.
• $solutions = ([xml](stsadm -o enumsolutions)).Solutions
• You know all that worthless unreadable XML returned by stsadm -o enumsolutions? Well, with this one-liner, we have turned the useless into something powerful.
• Installing a Fab 40 demo site collection programmatically - it takes a sweet long time to finish, but the script is painless and perfect. Others have posted batch files to install the templates, but no one's attempted to build all the demo sites automatically. This is a good example of what PowerShell can do.
• Enumerating over customized/unghosted pages—all of them in the entire farm—and getting a list of all pages that are listed as uncustomized. This is more useful than what the PRESCAN.EXE log gives you.
• On a related note, attempted to run the "RevertAll()" or similarly named function to revert all pages in the farm to default. Again this was on a copy of production data. And no, it didn't work as well as I'd hoped.
• Use CabLib.dll to create CAB files, bypassing the horrors of MAKECAB.EXE.
• Tiny development-time administrivia, like getting the four-part assembly name of your project's DLL, and copying it to the clipboard—you need it to declare in your feature.xml file for your Feature event receiver. Here's a bunch of things I did.

Conclusions

• You don't have to use PowerShell. I'm sure many of you have solved similar problems with PowerShell alternatives, including
• SQL queries
• WinForms utility apps (check out CodePlex, you'll see what I mean)
• Console apps
• custom stsadm extensions (I'm talking to you, Gary LaPointe)
• NAnt tasks/MSBuild tasks
• Telling people "that's not possible"
• It's hard to nail down for what I use PowerShell specifically, as you may infer from above. Is "lots of stuff" an okay answer? No?

Friday, June 06, 2008 8:00:01 AM UTC 

I recently read Todd Klindt's post on dynamically generating filenames based off of dates, for the greater purpose of making daily SharePoint backups.

USING BATCH FILES.

While his solution serves the #1 purpose of all scripts—it works, and thus is an excellent solution—I'd like to show how to accomplish the same thing using PowerShell.

Task #1: generate a SharePoint backup in YYMMDD format

DOS:

stsadm -o backup -url http://awesomeness.inc/ -filename "AwesomenessInc - %date:~-2%%date:~4,2%%date:~7,2%.backup.cab"

PowerShell:
(line breaks added for horizontal readability)

Task #2: delete backups older than seven days

DOS:

forfiles /d 7 /m *.backup.cab /c "cmd /c del @file"

PowerShell:
(line breaks added for horizontal readability)

Conclusions?

• Both get the job done.
• Date formatting in DOS is disgusting. forfiles is terse, but there's no excuse for the date formatting (substring selection?) syntax. This is almost as nasty as the fabled LDAP bitwise-and operator (look it up, it's not a joke).
• PowerShell is far more verbose (at least today), but benefits us with improved readability.

Had I the inclination, I could have used standard procedural techniques to further express my intent in the code itself and further improve readability. E.g. instead of calculating the YYMMDD date inline in the string, I could have called a function to do that for me, or stored the value in a variable, i.e. either Get-TodayInYyMmDdFormat or $dateInYyMmDdFormat. So while I did an okay job readability-wise, I could have done better.

As for DOS: this is as good as it gets, readability-wise.

Aside: calling DOS commands from PowerShell

Final quick reminder: instead of all that PowerShell-specific nuttiness, we could have instead satisfied the "run this in PowerShell" requirement by prefixing the existing DOS commands with "cmd /c". So the DOS command

stsadm -o backup -url http://awesomeness.inc/ -filename "AwesomenessInc - %date:~-2%%date:~4,2%%date:~7,2%.backup.cab"

becomes the PowerShell-compatible command

cmd /c "stsadm -o backup -url http://awesomeness.inc/ -filename ""AwesomenessInc - %date:~-2%%date:~4,2%%date:~7,2%.backup.cab"""

Note that we're treating the entire line after cmd /c as a string in PowerShell, so we have to be sure to escape any special characters, e.g. the double-quote. While it's somewhat obvious when I encapsulate the entire DOSO command in the string, it is much less obvious if you fail to do so. PowerShell will attempt to 1) evaluate what is there, then, if it fails, 2) send that value to DOS as text. If you don't specifically tell the interpreter "hey, send this as text", you may be surprised by the results. An example of this oddness is something I've used before.

DOS:

stsadm | find "feature"

PowerShell:

stsadm | find """feature"""

Thursday, June 05, 2008 4:49:58 AM UTC 

Those of you looking for something to amuse you will be sorely disappointed. Reading over this post even bored me, and I just finished writing it! Yeah, this one's for the search engines to pick up.

By posting this, I've become the world authority on Cisco NLBs with SharePoint

…which is sad.

I'm posting this partially as informational content, but partially because I'd be interested to find out if anyone else in the world searches for it.

So here goes.

Introduction: All SharePoint Tutorials Assume You're Using ISA Server

This is something that's annoying, especially in situations where they refer to ISA as "NLB". The SharePoint admin companion book is one place I can call out as guilty…

So, I'll point out before you go looking elsewhere that unless otherwise specified, the other guys' tutorial assumes ISA. You've been warned.

What I'm doing: please critique via comments

Since there is no posted content anywhere else in the universe, I'm just going to say what we're doing. This sort of thing is incredibly dry, so we'll do this lazy and concise, bullet-point style:

• New SharePoint "health check" web application, running on another (hopefully firewalled) port.
• Anonymous access is on. This is important! I don't think it's possible to have our Cisco NLB attempt NTLM, so we have to fall back on "nothing".
• This is not the same IIS Web Site as our main web application.
• Double-check to ensure IIS logging is off for this site. You don't need the logs, but whatever, you make the call, bigshot. Maybe those logs are more valuable than I will ever know.
• This is not even the same content database. I created a new (incredibly small) content database just to run the health check.
• Created a site collection, and
• Underneath the site collection, created a new blank site.
• Enabled anonymous access on the site - full access to the site, so that our health check actually works.
• Health check URL: /_layouts/mobile/mbllists.aspx - this is the where we're redirected if visit the mobile (./m) page of any site.Page looks roughly like:
   
• Tell the health check to
• visit this page
• expect a HTTP code 200 response (i.e. "it's good"). I don't think checking the page content is important, but if you want, knock yourself out with that.

Why it's important to go through all this trouble

I will say we've had trouble with our NLB health check because we (at the time) had not set it up to test every dependency. Our first attempt, a static HTML page, didn't even run on the same app pool as our main SharePoint web application…so when the app pool started throwing Out Of Memory exceptions, the NLB health check remained blissfully ignorant. Footnote: we fixed this problem by implementing overlapped recycling; definitely check it out if interested.

Even later, when running a static page on the same app pool, we were still not testing the database connection (and/or AD authentication!).

We also didn't want the health check page to be a "heavy" page (e.g. the portal home page). I briefly experimented with creating a minimal aspx page, but this "mobile view" serves the same purpose—a light page that pings the database. And hey, it's built in!

Feel free to give advice

If I'm missing something obvious, definitely let me know. If not, also definitely let me know :) I just don't know, and I'm not sure exactly where to look, so…blast away.

Wednesday, June 04, 2008 4:15:51 AM UTC 

 

 

The answer awaits you below the fold…

.

.

.

.

.

.

.

.

.

halfway…

.

.

.

.

.

.

Did you know that some feed readers give you a text-based preview of each post, and thus, if you're attempting to hide answers in your blog post by using whitespace…well…let's just say that this little meandering path, this seemingly useless little discussion we're having right now, serves a purpose.

Continuing…

.

.

.

.

.

Horrifying, correct answer: your SharePoint disaster recovery plan is stored on your SharePoint site! Uh-oh!

One act play

I've wrote a full-length, dramatic re-enactment of this situation, originally a 600-page manuscript. Which was way too big, so I ran Word Autosummarize on it, 40 times in a row. What you are left with is below:

 

Um…branding?

Also, if you answered "the problem with this SharePoint site is that it looks like SharePoint," then you're technically correct, unless your SharePoint site already looks like hawaiianair.com, or is in fact hawaiianair.com. If you're not running hawaiianair.com, by now you've already been told that lack of branding is a 'problem'. So, anyway, good job on guessing "branding," which is so commonly the answer that you might as well throw it out there every time you're lacking ideas. "Um…branding?" is always a good idea.

Lesson learned: off-site backups

I'm not going to be any more specific, because I don't have any magic automation to eliminate this problem—I'm just saying do something.

Tuesday, June 03, 2008 8:00:47 AM UTC 

UPDATE NOTES, 2009-02-02: I own an iPod now, and am now shackled to iTunes like the rest of you (or your Zune equivalent, for the other rest of you). Anyway, point being: I now realize this podcast client roundup no longer matters, because everyone is a slave to their mp3 player. I still think listening to podcasts on your car mp3 player on a long commute is a great idea, it's just that my commute is now 10 minutes as opposed to ~80 each way.

This will be painless and quick.

First, I'll point out that I'm a huge podcast consumer. I've listened to at least 500 hours of audio via podcasts in my daily commute, and have been doing so regularly since ~2005. I've installed every major podcast client, even Carl's Pwopcatcher, to see what works for me.

Criteria

I'm looking for a podcast client that will help me a) get a list of all current and past episodes of all my shows, b) download them, c) automatically and without fuss, d) so that I can play them in my car.

That's pretty much it. In the table below I'll add a feature checkmark for products that do exceptionally well at certain features. I'll also call out unique bad features in the table as well.

Podcast Client Roundup

Feature description	Juice	Doppler	iTunes	FeedStation
Automatically downloads new shows	YES	YES	YES
Permits you to download past shows	YES	YES	-YES-	YES
Consistently and successfully completes downloads	YES		YES	YES
Allows you to schedule downloads for later (e.g. 2AM)		YES
Bittorrent integration (for DNR)	YES	YES
iTunes podcast directory			YES
'I already have iTunes installed' bonus			YES
'I hate the crapware iTunes installs extra' bonus			YES
'Juice kills podcasters' bandwidth by redownloading'	YES
'Juice crashes more than it closes gracefully'	YES

Conclusions

Juice is the best client for most of us who:

• aren't satisfied with the little sandbox iTunes gives you, UPDATE: and don't own an iPod or Zune, which necessarily tie you to their corresponding podcast client
• can't be bothered to manually check each item for download (as FeedStation would have you do). I have 24 feeds, and I just synced, and it turned out to be ~180 new shows. Had I been using FeedStation, that would have been 180 manual "hey I like this episode, let's download it" clicks.
• cannot abide by Doppler's frequent failed downloads. Doppler, you guys are the best, MINUS this one big bad bug! Fix it and I'll switch!

Unfortunately, as is also stated above, Juice is the worst client for any podcast producer. I know for a fact I've downloaded the entire ARCast catalog 4 times (and it's huge). This is probably due to the way Juice stores and compares new podcasts with its podcast download history—the history is so finicky that whenever someone republishes an item in their RSS feed, whether it's to change the date published or even something a human wouldn't notice, chances are good that we (those running Juice clients) will re-download all republished items. Individually, this ends up costing Ron Jacobs (or whoever foots the bill anyway) $0.10 for another 1GB of extra bandwidth every time Juice "starts over" on his feed. Well, who cares, you say, it's only ten cents. Sure, it's ten cents for you, me, and the other 100,000 podcast listeners using Juice. That's a lot of dimes, and there's no end in sight.

So while Juice is the best podcast client available today, you'll never see any podcaster recommend it. They can't afford a 2x-10x jump in their bandwidth bill.

But, just as a secret between you and me (you being the 3 people reading this): use Juice. Or, if you can stomach the limited featurest of iTunes, do that. UPDATE: I've found my way around iTunes. It does let you download old shows, and plus, I own an iPod now, so I'm stuck with it, unless I want to do something crazy like use Songbird. I'm not crazy, so I don't use Songbird.

Friday, May 30, 2008 8:00:07 AM UTC 

Before I start, I'd like to acknowledge the fact that having strong opinions about an obscure Windows command-line utility is so not cool. I'm with you.

Things I don't like about using MAKECAB.EXE:

• If you forget to set an obscure property in your DDF manifest, MAKECAB will silently and happily exclude large files, despite the fact that you explicitly included them in the DDF manifest. Not only is this bad error handling, but it also gives you a false sense of security. This leads to the most awful troubleshooting sessions, where you have to dig into every layer of the stack, before coming back and double-checking every layer. This aggravating troubleshooting session is followed by a brief eureka moment, followed immediately by intense rage.
  Obscure property to set: .Set MaxDiskSize=<<some massive bytecount, I'd go 1 billion plus>>
  • Fun footnote: most online samples fail to set this obscure property, including MSDN and several blogs who obviously copied their DDFs from MSDN, which is why this is such a huge problem. Very few of the online examples get this right!
• The MAKECAB.EXE SDK document uses Times New Roman as its font. That dates it at pre-1997, when we were all still using either Office 6.0 or Office 95. Awesome. This isn't a complaint as much as it is an observation—OLD!
• Diamond Directive Files! DDF manifests are awful and, in my limited needs, completely unnecessary! Whenever I need to build a CAB file manually (it's happened a few times), I fired up PowerShell and used a single CabLib Extract/Compress function call to do the same thing that your no-longer-necessary-but-you'd-better-get-all-the-syntax-perfect MAKECAB+DDF file solution did.  And I didn't even have to bust out the ancient Times New Roman-sporting manual! (see below for actual script; it's tiny)

Takeaways

• Obscure property to set in your DDF manifest: .Set MaxDiskSize=<<some massive bytecount, I'd go 1 billion plus>> 
• If you're using MAKECAB.EXE as part of your Visual Studio build process, switch to WSPBuilder! You're not losing anything by swapping out one post-build command-line EXE for another, and will gain MUCH more in WSPBuilder's functionality.
• Use the following PowerShell snippet (or use it as a starting point for your own improved version) to compress cab files:

 

Copy-pasty version:

function Compress-Directory ($dir, $cabFileFullPathAndFilename)
{
    [void][reflection.assembly]::LoadFile(
        "C:\Program Files\WSPBuilder\CabLib.dll")
    $c = new-object CabLib.Compress
    $c.CompressFolder($dir, $cabFileFullPathAndFilename, $null, 0)
}

Compress-Directory "C:\temp" "C:\sandbox\a.cab"

Wednesday, May 28, 2008 8:00:52 AM UTC 

And by "dingo" I mean "SharePoint's PRIME API via stsadm -o import/export", and by "baby", I mean "links to as-yet-nonexistent pages in a wiki library."

So, running that back, after using SharePoint's stsadm -o import/export, you'll notice broken links in your wiki library.

Wow, that sounded much more exciting with dingoes and babies.

What are as-yet-nonexistent pages again?

Let's do this by example. Say, you're filling in content in a team knowledgebase, the one I like to call "the 2008 edition". You get a new one every year, but that's another topic, let's not get distracted.

So you're filling in the Widget page, and as you go, you realize you will also need to come back later and write up everything you know about Sprockets. So, in your page, as you are typing the word "Sprocket" you surround it with the standard wiki tags [[Sprocket]].

Up to this point everything is fine. You finish what you're doing on the Widget page, you log off, you go home. Tomorrow comes and you've forgotten about the poor lonely Sprocket page. And the day after arrives, and the day after, and so on, and poor [[Sprocket]] remains a link to an as-yet-nonexistent page.

So that's what they are.

What's the problem with this again?

I'll be honest, you'll probably never encounter this problem. But: if you're migrating content using SharePoint's stsadm -o export and then stsadm -o import, your "babies" will be "stolen" from you. Your [[Sprocket]] link will be stolen and replaced with some abominable hardcoded anchor tag that points to an error page.

So be on the lookout: wikis don't move nicely via stsadm -o import/export.

Tuesday, May 27, 2008 8:00:52 AM UTC 

Update 2009-03-05 - I've changed a bit

I've been meaning to update this post for a while, and here we are. Others have surfaced with interest

• Francis Cheung [MSFT] has written a similar piece outlining the challenges of unit testing SharePoint. He did a better job organizing his arguments, so go read his article instead of this one :)
• The SharePoint Guidance sample project was released—it's the closest thing we have to the "ideal* way to build SP apps*," and it exists, and you can download it today. And they have unit tests, and you can look at them. Awesome!
• Andrew Woodward has started a series about TDD in SharePoint, I'm interested to see this develop further with more focus on how to make it work in typical* SharePoint scenarios.
• Tom Clarkson posted an interesting article about testing his workflows. I stay away from Workflow development as a kind of metaphorical water hazard, but it's good to see testing Workflows is possible.
• Eric Shupps has posted a long contrarian piece on unit testing SharePoint. Be sure to read the comments for the counter-arguments, which boiled down to "your sample is too small to unit test anyway," AKA it's all about context. Perhaps the most interesting thing is that BIl Simser replied in the comments (comment #3) about how he structures his SharePoint projects, something I'd be interested in hearing more about.

*footnote: defining "ideal" and "app" and "typical scenario" is an exercise left for the reader, or the commenters on Eric Shupps' post.

Meanwhile, I'm spending my time learning coding/OO fundamentals, which is an important step to doing real TDD. Practicing TDD at home has already changed the way I structure my SharePoint projects, even the small ones, even when I'm not doing TDD at work.

I still don't unit test my SharePoint projects, but on my next project I'll probably try to do full-blown TDD. I still won't attempt to test code that interacts with SharePoint, but at least I can structure my code such that I can write down, and test, my intent. In other words, TDD can't tell me whether the listItem.Update() function is the correct way to save changes to a listItem, but it can tell me whether or not I called the function that fully intends to do so.

If you're coming here from the JOPX post, note that I do want to do TDD, and I think TDD has great value outside of SharePoint projects, but. I hold firm that most of the challenges we encounter with SharePoint are either the fault of the framework, or the fault of our misunderstanding of the framework—and neither is addressed by TDD.

I also want to clarify that as a rule, I don't think SharePoint should be used to host massive applications. If you've ever seen Wayne's World, there's a classic bit where his girlfriend gives him a present:

"A gun rack… a gun rack. Shyeah, Right! I don't even own "a" gun, let alone many guns that would necessitate an entire rack. What am I gonna do… with a gun rack?"

In this case, TDD on my tiny SharePoint projects is the metaphorical gun rack. Also big props to myself for making that connection, I deserve an award for that.

Opinion piece

This is my opinion, and reflects my (lack of) experiences, personal bent, and traumatic experiences, collected together into something coherent. I am writing this 2008-05-19, and reserve the right to change my mind later. I want to change. Like Fox Mulder, I WANT TO BELIEVE.

Authority

I have minimal authority on this subject and do not believe my opinion is "better". I am not an expert on unit testing, and I am not a renowned SharePoint authority. I'm going to attempt to steer clear of making any statements about TDD, and will instead focus on the more concrete act of unit testing.

I love comments

Actually I wouldn't know if I love comments, because I've only got like 3 of them. I was pretty pumped when I got those 3 comments though; that was a good year.

I love explicit disagreement

If you read this description of the forthcoming SharePoint Best Practices book, you'll see what I'm attempting to do: I'm attempting to start the conversation (hopefully not finish the conversation) that will answer the question: how are we supposed to adequately test our SharePoint projects? So far I have only found two strong opinions. I'd like to collect more.

Also, this is not a plug for the best practices book; I haven't read it.

Disambiguation: unit testing versus automated testing

One common misconception I'd like to clear up is that unit test probably doesn't mean what you think it does. Whenever someone tells you they are doing unit tests, allow me to present the following handy fact sheet:

• Just because something is written in NUnit or MbUnit or xUnit.Net, does not mean it is a unit test. I think this is the #1 source of confusion—the name of the product has the word "unit" in it, and it's used for testing, so…unit plus testing, unit testing! That's what they're called! Hilariously, by this metric MSTest is the best-named automated testing suite. You heard it here first: MSTest is the best!
• TFS Web tests are not unit tests. Tests that involve WATIN or some other web automation tool are not unit tests.
• Anytime you're actually newing up an SPSite or SPWeb object inside your test method, it's not a unit test.

Now for a few facts:

• Unit testing is often confused for the broader world of automated tests, which includes both unit tests and integration tests.
• Unit testing is not easy in SharePoint.

At this point I'm going to be lazy and tell you to read up on the Wikipedia articles on Unit tests and Integration tests I just found by googling for 5 seconds. I'm lazy, ask someone else for a better explanation.

Opinions

Popular culture: no testing

When dealing with such a broad topic as SharePoint testing strategies, it's probably best to start with the default. Which is to say, nothing at all. By default, the out-of-the-box Visual Studio 2005 projects do not create a separate testing project for you to use for testing. Most developers will live with the defaults, which is to say again: nothing.

So if you're here, wondering what to do outside of the defaults, congratulations! You're already counter-culture.

Counter-culture: agreement on some testing, disagreement on specifics

(various): Integration testing is sufficient

Spence Harbar, MVP: Unit tests with TypeMock - this is the only page I can find from him on the subject.

Casey Charlton: Unit tests against Doubler-generated wrappers - Casey has written repeatedly about the pains of his unit testing approach, but the fact of the matter is, he now has a workable, no-TypeMock-required solution for unit testing in SharePoint. Check out the Doubler addin to Reflector, which is described as "useful when working with legacy code."

My Opinion

With the problems I encounter on a day-to-day basis, I don't think it's worth it to go out of my way to build unit tests. And the reason is as such: when I encounter a bug in my SharePoint application, it has nothing to do with my code, and everything to do with my interactions with the SharePoint framework. Let's do this by example:

• Spence Harbar on his TDD page mentioned specifically the awful quality of code in the navigation controls he's seen in the wild. His assertion is that the code quality is the biggest pain point. Unfortunately, when I'm attempting to build something, I'm worried less about the cyclomatic complexity or the C.R.A.P. metric (it's real, believe it), and worried more about which of the many portal-querying strategies to use. There's no way to know if you're using the right approach, unless you're read all the books, all the whitepapers, and all the blogs. And, yeah, don't look at actual control vendors' code via Reflector, because apparently they're doing it wrong too :) The major issue when building navigation controls is not traditional code quality, as much as a broad understanding of all the options presented to you by SharePoint's object model.
• In my most recent project, I encountered a race condition where the ItemAdded() event fired too early and my list item…did not exist yet. This was nothing I could control. Solution? What I affectionately refer to as the "try try again" pattern. It involves a while loop with a retry counter, a Thread.Sleep(), and a whole lot of swallowed pride. Feel free to complain about the solution, but don't complain to me, complain to Steven Van de Craen; I'm just thankful his solution works. This concurrency bug was out of my control and would not have been discovered (or fixed) by unit tests.
• Outstanding issue: I am intermittently faililng to update my item after the ItemAdded() event. There are several possible solutions; the two I will be looking into will be a) disabling event firing (shouldn't have been a problem); b) following this awesomely-detailed advice and replacing my ".Update();" call with a ".SystemUpdate(false);" call. Either way, proper testing would not have saved me from this pain; detailed (esoteric) knowledge of the framework is the only thing that could have saved me.

What I'm saying is that practicing TDD in my SharePoint projects would not help me improve my code enough to be worthwhile. In the meantime, I will use automated tests (mostly integration tests).

While I'm not especially pleased with it, I am resigned to avoid unit testing the difficult pieces of my SharePoint projects. I'll consider changing this stance in any of the following situations:

• I become a master ninja and TDD becomes more effortless to me than doing without;
• I'm working on a critical, high-impact, risky project and need any and all available methods of testing and program verification;
• I have a lot of code in my project (which begs the question: why?);
• I learn enough about SharePoint development that situations like the three above are no longer issues, i.e. I learn how to walk around the coding landmines and am confident with every interaction with the framework. (For now: BOOM)

Don't forget about your team

Another factor to consider is maintenance.  Remembering that by default no one builds automated tests, with some effort counter-culturists write integration tests, and only two people in the world practice TDD in SharePoint—it's a hard sell to say "hey, I want to try building this next SharePoint project into a structure that is so awful and alien, so overwhelming, that you will be paralyzed. I'm going to change the project into a gorgon, which, with a single gaze, will turn you to stone where you sit. You will be completely unable to function. Do you mind? This new method will allow me to capture up to 60% of the bugs. But probably a lot less than 60%, to be honest. TIA!"

Wednesday, May 21, 2008 8:00:52 PM UTC 

Update: I used the word 'connection' in the title of this post for a reason

To everyone who links to this post with the phrase "SharePoint is written in Python", please allow me to make it super clear: no it isn't. The commerce module of a spiritual ancestor of sorts of SharePoint's was written in Python. That was Site Server.

To make it more clear: SharePoint doesn't have a commerce module. Commerce Server does.

Sorry I felt I had to do this, but, so many people were linking to this post with the semi-twisted-truth ("Did you know that SharePoint was written in Python?") that I will risk ruining this fun little history lesson by spoiling the ending. Sorry; on with the show.

Fun update: Greg Stein has written an article expounding on all this: Microsoft Ships Python code…in 1996

Original Introduction

Before you ask, no, this isn't a sneak peek into SharePoint vNext*. Instead, we are going to visit the ghost of Christmas Past:

FLOSS Weekly 28: Greg Stein for WebDAV

* However, if you can hook me up with any kind of sneak peek into SharePoint vNext, I'll pay you, cash**. Or, as good as cash anyway: old SharePoint books. Those things are timeless! Your knowledge never expires! (see depressing footnote at end of this post)
** Contrary to what has just been written, we don't endorse crime

Know your history

This discussion will be meaningless if you're not aware of SharePoint's origins. This SharePoint history page has an excellent summary, and even more excellent SharePoint history diagram. Go go Visio skills, good job sirma'am!

Anyway, if you're interested, click through the diagram below.

Let's talk to Greg Stein

From FLOSS Weekly 28, at the 47:30 mark:

SharePoint! Python! Scandal!

To spell it out for you: Site Server, the product which eventually became "Tahoe", which eventually became SharePoint as we know it, was written (at least in part) in Python! Python! In 1996! In 1996, a predecessor to SharePoint was implemented partly in Python!

I didn't believe it the first time I heard it, but there it is. SharePoint's predecessor was for a brief moment in 1996, a pretty cool web application written in Python. Sounds a lot like 2008, doesn't it?

 

Attribution: the diagram is not mine, though the offensively obnoxious green text is mine.

Depressing footnote: learning investments

Somewhere around the ~38:20 mark in the podcast, they ask him "what is your favorite programming language" and he replies that he has been programming in Python for 13 years, and using Emacs for 22 years.

I'll leave you for a while to ponder the incredible longevity of his ("the other guys") platform.

To me this news is incredibly depressing—how long have my programming language/platform choices been valid? How long will my SharePoint skills remain useful to any extent? I don't want to be on the Microsoft learning treadmill forever if I'm churning major parts of the framework every 2 years. Can you imagine evolving your ecosystem for 13 years? Can you imagine learning your IDE in-and-out, and continuing to use it for 22 years? It's almost unthinkable in our (Microsoft) ecosystem.

Argh, I'll stop. 13 years! 22 years! And counting!

Tuesday, May 20, 2008 8:00:39 PM UTC 

This last Saturday I attended the Austin Code Camp 2008. I didn't take notes so am unable to post coherent bullet point recaps of each session; instead I will post what must be, by logical extension, incoherent.

The Krewe of Austin

Something is noticeably different between the various Houston community events and the code camp I attended in Austin. A few notable things about this code camp that I liked:

• The code camp did not feature an "Introduction to Silverlight" session. This is the simple, effective test by which I will measure all future conferences: is there an intro to Silverlight session. If there is, then good chances are most of the sessions will be useless. Further down the road, change out "Silverlight" with whatever new UI framework/data grid/designer tool that is "up to two years away from release." I'm officially tired of these type of talks, 4 LIFE. Call me back when you're running a "Best Practices in Silverlight Smackdown."
• The code camp was heavy on OO principles. This is a good thing; between "patterns cage match" and "IoC jumpstart" and "OO design" and "mocks and stubs", my brain was assaulted by lots of OO. Which is good, I haven't gotten this much in the 3+ years of involvement with Houston user groups and events. Not to gripe on my home city, let's stay positive etc.
• The remaining slots were filled by oddball (but useful oddball) sessions—in particular I liked our "Sarbanes Oaxley fishbowl talk" with heavy audience participation. I also liked the advanced SharePoint session, which helped me identify two major bugs in my project that I discovered by asking about the presenter's 12 lines of code. Let me repeat this for emphasis: he only had time to show the barest minimum of code, and yet I managed to pick up two places I need to change my own code, from that tiny snippet shown for just a few minutes. Sigh, SharePoint is hard sometimes.
• Informal and efficient. You kind of showed up, you picked up a drink in the hall if you needed one, the presenters passed out books at the end if they remembered, registration took no longer than any other user group meeting. Self-service over concierge; it worked; awesome.

#1 benefit of attending: INSPIRATION

As sarcastic as I usually am, I want to let you, my dear reader, to know that the following is genuine and there will be no punchline involving abhorrent use of embedded MIDI and/or MARQUEE tags and BLINK tags that still work in Firefox for some reason.

But really, I did get one thing out of the code camp that I can't say I get anywhere else: I left feeling inspired to get awesome. I don't know at what precisely, I don't think it matters. What matters is that I've got the energy now. I'm motivated.

Yeah, now to act on newfound motivation—noted. Track record: not good; will try anyway.

So what do we take away from this rambling incoherent post?

1. Thanks to the organizers/speakers of the code camp, it was awesome and at least one person (me) enjoyed it. I would like to place an order for ONE MILLION MORE of these camps; please schedule at your earliest convenience.
2. I'm looking forward to the Houston Techfest 2008 [placeholder link; website not updated from 2007]. If anyone reading this a) also hates intro sessions, b) is willing, and c) is competent enough to be an authority on a given topic, please submit your session to the Houston Techfest organizers. You can even be from Houston, you don't even have to be from Austin to give a good presentation. It's great how that works.

Friday, May 16, 2008 7:19:58 AM UTC 

I am sometimes asked what PowerShell is. While it's easy enough to give a technical overview, today I'll focus on everything else.

PowerShell is not a fly-by-night scripting tool

It isn't JScript, and won't disappear anytime soon. For the three JScript enthusiasts remaining in the world, I apologize for the cheap shot. Wait, no I don't. It's JScript—no one's using it, no one will be offended.

PowerShell has had a rapid rise since it reached beta status in late 2006, and if you're wondering if this positive trend will continue, let me lay it out for you eloquently:

WHY YES IT WILL, AS A MATTER OF FACT

You may have already heard that Exchange 2007 uses PowerShell for its administration console. Which is great, but I personally couldn't care less—I've got bigger fish to fry.

SQL 2008: PowerShell support. Still don't care, but we're getting warmer.

IIS 7: PowerShell provider. Ok, I like this. Still in beta though, we'll give it some time before any "visible enthusiasm."

administrative duties for Microsoft products going forward - pretend the link to the left is from an authoritative source. Because had I better searching skills, it would be. The point is: major Microsoft server products will begin to support PowerShell in the same way as Exchange 2007 (which again, we couldn't care less about).

If all that hasn't convinced you, maybe the visual aid below will:

 

In Windows Server 2008, PowerShell is an available Feature! We're beyond installers; PowerShell is now built into the operating system! This is even true of the older Operating Systems—if you don't believe me, try uninstalling PowerShell! Can't find it? That's because (in the older OSes anyway) it's installed as an update to Windows, as a standalone hotfix. Hotfix, hotness! If I was rapping I could make "hotfix" and "hotness" rhyme. I could  pull it off.

PowerShell is not going to replace the command shell (cmd.exe)

I briefly thought this would be the case, then Server 2008 Core dashed my hopes. While I admire the engineering-won-over-marketing victory that allowed Server Core to ship with almost absolutely nothing, including all of the .NET framework, I wish we could have simply declared 2008 as "year of the PowerShell" and Server 2008 as "the painful version for the PowerShell laggards." That would have been awesome.

But, as it turns out, Server Core did indeed ship without PowerShell, and as such, we still have cmd.exe available to us, awful parsing rules and all (e.g. try the following: echo "greater than sign is >" - I assure you that will not produce the desired effect). Anyway.

You are not wasting your time learning PowerShell

First, I'll point out that PowerShell is actually quite easy to pick up, and that you can do so with a single PowerShell book. Furthermore, I'll point out that the currently-in-pre-alpha-V2 version of PowerShell is additive and does not contain any breaking changes. Like C#, PowerShell will grow, only. And grow, awesome!

PowerShell doesn't have to be slow

I've read that the PowerShell process will be slow loading the first few times. Which is terrible, because many of us won't make it past those first few times, especially those of us who are cynical of the whole thing to begin with. I assure you, it gets faster.

PowerShell is useful

I'm not telling you to pick up PowerShell to satisfy your yearly programming language learning quota. I'm also not telling you PowerShell will change the way you code in Java or COBOL or SNOBOL or FORTRAN or RPG or whatever. I will instead say that I have saved myself time (and more importantly, aggravation) with PowerShell, whether by spelunking the SharePoint object model, completely automating tasks, or running scripts as post-build actions on my Visual Studio projects…

PowerShell saves me time. And if it doesn't save me time, then it's saving my sanity. And if it isn't doing either of those two things, then it's probably pulling in some mixture of .NET framework classes, using enhanced filesystem/XML/registry support, AD stuff, COM objects, and yes, even running other console applications—it's pulling any or all of these things into one environment, in a way you won't see anywhere else. PowerShell is the de-facto scripting environment for Windows—in fact I used to call it "Perl for Windows, except Perl for Windows already exists, and actually competes directly with PowerShell." Yes, I can't help myself when describing PowerShell and say the quote in full, all the way to the end of the long rambling sentence (like this one (it doesn't seem to ever end)). Well, let me set the record straight as to PowerShell and Perl:

PowerShell is better than Perl

There, I said it.

Well, for Windows, anyway.

Thursday, May 08, 2008 8:00:07 PM UTC 

UPDATE 2009-09-01: People Are Apparently Still Finding And (presumably) Reading This Outdated Article

First: I'm sorry. In the post below, I took a fun little journey comparing SharePoint's built-in wiki with MediaWiki. After some first-hand pain with wikis and reading the comments I received on the post, allow me to flip-flop my position and clearly state: SharePoint 2007's built-in wiki will impede any attempt to build a real wiki.

To put this statement in context, your main problems when deploying a real wiki will be to get critical mass. This means, you'll want to get hundreds or thousands (or tens) of individual contributors on your wiki. And you won't want them getting frustrated with the software. You don't want them trying to figure out the 4-step process to uploading an image; you don't want them attempting to build an HTML table (or anything really) using the source view; you don't want them to lose data when someone 'deletes' a page to the Recycle Bin and no one notices for 4 months. Preferably, you don't want them thinking about anything but 'how do I best contribute to this group knowledgebase?' And SharePoint will get in their way.

So, my new statement is: if you need MediaWiki, or something like MediaWiki, invest the time and effort to do so. Don't try to build on top of the minimalist SharePoint wiki. But…

But, with the most basic of needs, and given the assumption you already have SharePoint, and that Integrated Windows Authentication works for everybody, SharePoint is your best bet. Once you start doing 'fancy' things like adding images or attempting to go beyond the built-in WYSIWYG editor, you'll experience pain. Even with some pain, SharePoint's a good choice for small wikis. The original article (starting in the next section) lays out this argument in heavy detail.

Second: The new SharePoint will be out soon, invalidating whatever points I made in this article. Consider yourself on notice: this article is about the 2007 version, folks.

Third: I'm really trying to stop writing these "op-ed" pieces that really don't help anyone. I don't think I introduced any valuable concepts here, just pushed a lot of hot air into the atmosphere. How does this article (and those like it) help? They don't. So, I'll stop.

Final note: I've also flip-flopped on Wiki syntax. It's not that bad after all. WYSIWYG is much more important than markup. And while you're at it, everyone just copy PBWorks, because they have a sweet setup.

Original Introduction

A while back I came across this direct comparison of MediaWiki, the wiki engine powering Wikipedia, and SharePoint's built-in wikis.

As you might imagine, the comparison favored MediaWiki. And, as I left an embarrassingly passionate comment rebuttal (in which I link directly back to this address), I feel compelled to write the following post.

Summary first: because I meander quite a bit

The summary is: you probably don't need MediaWiki's extensive featureset for your corporate intranet. Assuming limited needs, SharePoint's wikis are a great, no-maintenance, small-learning curve solution to the "informal team wiki" problem. You can do better feature-wise, but why would you want to? Plus, you already have SharePoint—this is a no-brainer.

Well, let's get with the details.

Wiki syntax: unnecessary

So, let me start off by saying I hate wiki syntax. Actually I need to clarify: I hate any markup syntax built on top of HTML, and that includes MediaWiki's wiki syntax, the FlexWiki syntax I've used, whatever syntax ScrewTurn Wiki uses (it's different from FlexWiki), and the perennial favorite, [BBcode] markup. I hate all this for two reasons:

1. It's annoying to learn a new syntax for bolding up a word when <strong></strong> works just fine in HTML thank you, and
2. It's totally unnecessary.

We are now presented with the question: but Peter, what alternative do we have?

Just use HTML

Just use HTML! Allow me clarify in the next section:

You can parse the input and allow only a subset of HTML

I'm not saying to throw wide the gates and allow everything. It's perfectly reasonable to parse input and allow only specific markup, negating any and all security concerns! And when I say "parse", I don't mean some kind of abomination involving substring searches or even regex.

So if someone types: "Hey, I like to <strong>bold me up some text!</strong><script><!— some nasty js virus bomb —></script>", you know what? You can figure out which parts are legitimate and which parts are no-no! Don't just give up and say "oh forget it, let's just change the <b> to [b] and make our 1 billion users learn this new, inferior syntax." You know what? YOU CAN FIX IT, DON'T GIVE UP! 

It's possible; look into it—it's better than inventing Yet Another Markup Language. [[Insert your own awesome YAML joke here!! Ha ha, good one!]]*
* for the purposes of this bit, YAML stands for "Yet Another Markup Language"

Back to wikis

So with Peter's Fundamental Law #14 ("Don't invent yet another YAML") out of the way, let's get back to the comparison.

It's all about context

What are you looking for in a wiki—are you attempting to build "wikipedia for my global enterprise"? If so…I don't actually believe anyone's attempting to do this. Leave me a comment if you are, and if you have implemented MediaWiki in a corporate environment to accomplish this goal. There probably are people attempting to do this, which is fine. I haven't met them, but I'm sure they're out there. Maybe.

But I'm here today to tell you that most of you don't need MediaWiki. You may think you do, but if you actually go out and talk to some of your users, you'll find out: you don't. Your corporation is not filled with anonymous trolls—the trolls are still around, but [[insert your own boss joke here]]. Ow, my sides!

Anyway, MediaWiki is built to work with the semi-anonymous internet, and on a massive scale. You, on the other hand, have a grand total of 30 people in your enterprise who are clued into the fact that the mouse sitting on their desk has a right mouse button, and have a solid understanding of how a folder can contain other folders, and files. There's 30 of you.

The corporate intranet is not the same as the raw, unfiltered wilderness of crazy that is the internet.

What crazy world do you live in anyway

Let's just state this as a fact: your IT department does not have linux admins. Or if they do, they're hidden so well you may not recognize them. The point being: when you ask someone to install MediaWiki, you'll be asked: does this thing run on Server 2003, or does it require Server 2008? Um, gusty* gibbon or newer? That sort of answer will not register; please try again.

Oh, and while we're talking corporate IT politics, you may want to avoid using the words "Open Source" or "free" or "third-party" or "custom" or pretty much anything. Also, try and somehow pretend MediaWiki is made by Microsoft. If you say the wrong word, or fail to say the correct ones, you may alert your IT group to the fact that this software is unapproved!

* this is hilarious, think about it

Who becomes the admin?

Something else to keep in mind: if IT actually goes off and actually installs the thing, how are you ever going to get access to it? If you haven't learned by now, the easiest application to support is the one with the fewest users. And a good approach to keeping your user count down is to prevent them from crowding in too much in the first place. Zero is the best number!

SharePoint Time

Let's get down to business: actually comparing SharePoint's wikis to MediaWiki, head to head. I'll take the SharePoint perspective:

SharePoint PROS:

• SharePoint wikis take a grand total of two minutes to install and configure. Two minutes.
  • Infrastructure concerns, like high availability and security, are covered.
  • As Don King would say, it's "supportatudinous!" IT supports it already! It is backed up like the rest of your SharePoint data! You'll get upgrades for free!
  • You didn't even have to fill out the 8 page online help desk form, followed by a 10 page project request form, followed by a 3 page departmental project request form, followed by 2 security forms, followed by a third "surprise!" security form. You just made the wiki, all by yourself! Like a grownup! It was so simple I didn't even have to use fancy words like "provision"!
• SharePoint wikis use Integrated Windows Authentication*, emphasis on the word Integrated. The point is: seamless security. Footnote: security is configurable, maybe you're not using Integrated Windows Authentication specifically …but you should be.
• SharePoint wikis have minimal wiki syntax, instead relying on the rich text edit control. What's the syntax to bold up some text? Oh yeah, click the bold icon. Awesome. *tiny footnote: many wiki implementations have a WYSIWYG edit control, not MediaWiki specifically.

SharePoint CONS:

• SharePoint wikis are relatively feature-thin. I'm totally okay with this, but I'm sure the grizzled internet wiki veteran will be angered by, for example, SharePoint's extremely easy-to-delete pages, and lack of discussion pages, etc.
• SharePoint wikis are lists, and as such, should probably remain smaller than 2000 items. Yeah, there's a whitepaper on the subject; after too many items, performance of operations on a list slows to a crawl.
• SharePoint costs money, somewhere down the line—either for SharePoint itself, or for SQL Server, or just the server license, SOMEWHERE down the line, you're paying.

Intranet NOT Internet

If there's one thing you get through your skull—actually, if there's one thing, let's make it Peter's Fundamental Law #14 ("Don't invent yet another YAML")—okay, let's try for two things. If we have room for two things, the second one should be: the intranet is not the same as the internet. When comparing wiki engines, keep this in mind! The intranet is not the same as the internet!

Monday, May 05, 2008 8:00:29 PM UTC 

I just fixed an odd 403.14 server error on my Server 2008 VM, and what was the problem? My application was set to the "DefaultAppPool" app pool. The cause? "DefaultAppPool" doesn't exist, and yet my web site is somehow set to run on "DefaultAppPool". "DefaultAppPool" didn't event exist at the time I created the web application in IIS, much less now. By all means this should not have been a problem—IIS should set newly provisioned web sites to an existing, non-imaginary app pool—yet it was a problem.

So, advice: don't delete the default app pool, even in the new hotness that is IIS 7 and Windows Server 2008. You certainly can do so—I'm not saying it's impossible to get along without the default app pool; I'm just saying that you'll have surprises. And not the "congratulations, it's the prize patrol" type surprises, more like the "congratulations, your vehicle has been towed!" surprises.

You don't have to take my word for it—instead listen to Joel Oleson (talking about IIS6), and I quote: