Peter Seale's weblog

Wednesday, October 28, 2009 7:01:27 AM UTC 

I think now's a good time to close out the SharePoint tag on this blog, marking the end of SharePoint 2007-focused content. I'm creating this post as a sort of table of contents for my SharePoint content. I'll attempt to group posts into themes and then editorialize. Some of my earlier posts I'll even recant! Here goes.

Things I think anyone (SharePoint community or otherwise) would find interesting or useful

• Thinking Creatively - what I consider possibly my best SharePoint-related post, because it contains transferable concepts. The idea is that we as developers must go beyond our traditional code monkey role and do some critical thinking while specifying/designing solutions to problems. This is illustrated with an excellent story told during an Agile conference session. Also I recommend the linked Agile Toolkit podcast episode that inspired my post.
• The SharePoint-Python connection - if anyone ever tells you "SharePoint was written in Python", I apologize. Anyway, misquoting aside, this is a fun little bit of history.
• Golden Rule of Troubleshooting - I misdiagnosed an error, posted the erroneous diagnosis to my blog, and to save face hurriedly changed the contents of this post to be about troubleshooting. The golden rule of troubleshooting, for those of you too lazy to click through: beware the invisible proxy! It takes many forms! It strikes silently! Everyone will think you're crazy when you tell them network gremlins are eating your incoming packets, but leaving your outgoing packets alone!

Hilarity

• Welcome to SharePoint - a real-life nightmare scenario I encountered while troubleshooting a SharePoint 2003 "desktop issue." It turns out, the 15 pages of IE settings + Active Directory group policy + various Office ActiveX controls + virus scanners + IE version mix + network security appliances + Kerberos + firewalls + IE Zone settings + DNS/DHCP issues + AD replication issues + expired password issues + routing errors + spammy IE toolbars…any and all of these things, if out of whack, show the same "username/password" dialog. The post was a joke, but after troubleshooting every flavor of this problem, it gets to you a little. Anyway, welcome!
• What's wrong with this picture? - mildly amusing scenario involving disaster recovery documentation. Trust me, this is as hilarious as disaster recovery documentation is going to get.
• STSADM: Spot the typo - a lament for the endash. This is as much hilarity as you'll find on the topic of Word AutoCorrect.

Frustration

• Angry at CAML - I remember writing this post after a three days of wrestling with GetListItems, most of which was wasted learning idiosyncrasies. I then deleted most of the unhelpful angry comments, so what remains is the milder parts. This was my first "surprised by how difficult it is compared to how easy everyone makes it sound" experience. Visit for a link to the greatest Oracle DBA ever. Or visit for my graphical representation of the MSDN Rage Meter.
• I'm like, angry at numbers - in burnout mode, and ranting. If there's anything to take away from this, it's a) keep a sense of perspective (i.e. this stuff isn't important), and b) don't invest time in new Microsoft frameworks as a rule.
• Disposing SharePoint Objects: Survival Mode - this was a tipping point wherein I realized that no one does SharePoint development properly, even most of the MVPs. Keith eventually discovered and wrote down all of the disposal rules, and from there someone at Microsoft released SPDisposeCheck (which I believe covers most scenarios). Anyway, the subject of disposing is now a moot point—the more interesting bit is that, as of two full years after RTM, we had incorrect guidance as to how to dispose ~2MB objects on web servers typically running a maximum ~1000MB in the worker process. Sort of an eye-opener.
• Beyond technical challenges - rant, wherein I close the SharePoint 2007 portion of my blog (oops—the ban lasted a full month anyway, until I couldn't hold out). There are some takeaways here, notably that everyone's struggling with SharePoint, including the MVPs and "experts." I make the statement that every person working with SharePoint should look beyond their immediate technical challenge and ask, is SharePoint the right solution? Also, I challenge the assumption that SharePoint is a good developer platform.

SharePoint as an app dev platform

• [also referenced in the "useful" section above] Thinking Creatively - what I consider possibly my best SharePoint-related post, because it contains transferable concepts. The idea is that we as developers must go beyond our traditional code monkey role and do some critical thinking while specifying/designing solutions to problems. This is illustrated with an excellent story told during an Agile conference session. Also I recommend the linked Agile Toolkit podcast episode that inspired my post.
• Argue with your customer - I think I posted this after failing to convince my customer to go with less SharePoint customization and more out-of-the-box features. I still get a lot of pushback when I try to prevent SharePoint customizations. If there's something to take away from this, especially as a non-SharePoint developer, it's that not all features cost the same, and not all customizations cost the same. Making the relative costs clear to your customer should (should!) encourage them to avoid the more costly customizations. I'm always shocked at how someone will tell you "no, we need it to work exactly as I've told you!" and then turn around settle for a vendor product that does about half of what they want, but costs more.
• 80%, then stop - wherein I tell a story about my experience with the Pareto principle as it applies to SharePoint solutions. Also: you don't have to write your apps in SharePoint! If it doesn't save time, and you don't know of any benefit you'll gain from using SharePoint, then why are you attempting to use it?
• Estimating SharePoint tasks: cry for help - scary realization that I'm still unable to estimate how long something's going to take, primarily because I'm constantly blazing new (new to me?) trails in the SharePoint API, making bad assumptions about its behavior, triggering bugs, and running into unexpected limitations. Any of these things can cause multi-day delays. It does get better if you're writing a second or third app that deals with the same part of the API.

Framework limitations or errors

• How many is too many [SharePoint list] items? - the SharePoint whitepaper announcing the 2000 (now 3000) item limit per container was something of a blow. To say it clearly: this limitation prevents you from using OOB lists for anything with real traffic. There are over 3000 days in 10 years, so at 1 item added a day, you're running into the recommended limit. Since then I've seen some crazy errors with large lists, mostly revolving around OutOfMemory errors, crawl errors, using PRIME-derived features on the lists, exporting to Excel, breaking the Grid-view, and so on. So the list size limitation is real, if not a "hard limit."
• SharePoint Workflow Nuttiness, Volume 1 - My initial foray into SharePoint Workflow development ended in pain, where I had to scrap an entire approach because Workflow doesn't support state machines with replicator activites. Then I read Ayende's JFHCI series and it poisoned me forever against WF. I wonder now what problem Workflow attempts to solve, and why don't we just use a pure code solution instead? Note that Ayende wrote a book on DSLs (Workflow is a form of a DSL), so don't just pretend he's some crank with a blog*. Final note: this is one of my top 3 most visited posts, so apparently lots of people have run into the specific issue.
  *I'm aware that by definition, I'm a crank with a blog
• Illegitimate ErrorWebParts - a crazy solution to a crazy problem—here I use the "crank the chainsaw a few times" metaphor to describe loading a SPLimitedWebPartManager. Really, this is bad.
• Dingoes stole my babies! - wherein I discuss a problem with moving wiki content via the PRIME API.
• SharePoint awesomeness: User Profiles - wherein I discuss a potential benefit of functioning User Profiles. Unfortunately this post was premature, because the scenario I envisioned/laid out in the post wasn't possible out-of-the-box. Oops! Another framework limitation.
• SharePointPdfIcon project - wherein I announce my (failed) CodePlex project. It works great for single-server farms, incidentally. I just can't be bothered to spend the time to write all the timer job junk to make it work on multi-server farms when even this souped-up solution won't work when someone adds a new server to the farm after activating the Feature…this is one of those cases where using the SharePoint deployment framework causes more pain than deploying changes the "vanilla" way. Ugh.

PowerShell + SharePoint

• Find the DLLs - After determining that Lutz' (now RedGate) Reflector is a core tool for SharePoint development, the next step was acquiring the DLLs from wherever they lay. Enter gratuitous use of PowerShell to solve the problem.
• PowerShell is Magic: Part 1 - wherein I demonstrate PowerShell calling STSADM but also calculating on the fly. PowerShell is really, really useful.
• PowerShell is Magic: Part 2 - wherein I describe (poorly) how the PowerShell REPL is powerful.
• SP + PS: Working with wikis - wherein I give a pretty weak (but like the movies say, based on true events) example of how I use PowerShell to solve problems.
• Why PowerShell: Readability - wherein I take another shot at explaining how PowerShell launches processes (console apps)…and get the explanation wrong. I should probably post an update or something. Also, PowerShell can be made to be readable (though like Perl can be made to be abomination).
• PS + SP: Cornucopia - wherein I list all the real-world uses I've found for PowerShell working with SharePoint. PowerShell is uniquely useful for SharePoint, because SharePoint has a) an incomplete admin UI, b) huge object model that's loaded into the GAC, c) incomplete MSDN documentation, necessitating experimentation, and d) so much XML! Probably other reasons, but those are the big ones. Also, Visual Studio post-build tasks are the devil. I'm now ashamed of me of a year ago. Shame on you, 2008-me.
• Useful PowerShell functions I've written. I've looked at others' PowerShell functions and I think it's a lot simpler to do away with logging, comments, object disposal, and attempts to improve performance. All things are appropriate in context, but for me, these are mostly throwaway ad-hoc scripts, and are thus simple and focused.
• Write-ListDetails - particularly, discovering (and recording useful information about) large lists—and remember, this is PowerShell, so you can pull ANY data you want, no matter how complex the criteria or where the data originates.
• Run-Query - think of this as a REPL for your SharePoint Enterprise Search SQL queries. Returns pretty objects, not a DataTable.
• Get-CrawlHealth - I used this to prototype the functionality I wanted, then built it into a _layouts page. The script works though (with the exception of the $contentSource.CrawlCompleted property, which is inaccurate and worthless)
• Update-SearchScopes - on demand! You can't do this via the UI.
• Get-UserProfile and Get-UserProfileData - the first function retrieves the UserProfile object, the second function maps the (nigh-impenetrable) property collection to real properties. Useful for bulk data export and for examining your user profile data in a meaningful way.

Informational (knowledge, not concepts)

• Briefest introduction to GetListItems using CAML and lists.asmx - by now there are much better (and more accurate) guides to GetListItems. What may be amusing to you is the comments I leave on each line of code—wherein I document how uncertain I am of what each element does. The MSDN documentation has improved since 2007. As a small bonus, I'll note that this runs against WSSv2, not SharePoint 2007.
• Don't delete the default app pool - nitty-gritty details on IIS configuration. Note to anyone who has rolled out a SharePoint farm: congratulations, you're now qualified to roll out any ASP.NET app! Personally I'm pumped this knowledge transfers.
• Firefox supports auto-NTLM logins - most of you aren't aware that you can use Firefox and visit your SharePoint sites, and not be so aggravated by login boxes—Firefox supports automatic NTLM authentication in a manner similar to IE! Follow the directions to enable it.
• Cisco NLB setup in SharePoint - because I'm still the only resource for this in the entire world. Ridiculous.
• SharePoint search - faster than you think - wherein I complain about how slow IE is and how it is to blame for many of SharePoint's "performance issues." Honestly, it's true—try loading SharePoint pages in Firefox, they're way faster. Also it helps if the page doesn't load 1MB of JavaScript and another 1MB of inline style text.
• SharePoint Timer Jobs - here I attempt to shift from unhelpful ranting, to a post designed to help others avoid pain. I'm happy to say this is one of the top 3 posts, and hopefully it's helping people. Specifically, I mention that Timer Job updates require the manual reset of each Timer service on each server, and provide a script to quickly reschedule a timer job. Small footnote: I would rewrite the PowerShell script today such that it was a single function that takes arguments instead of requiring customization of the script. Functions are self-contained, and can easily be pasted into a PowerShell window (e.g. a PS window running remotely on a server!) without accidentally executing anything.
• Project retrospective on my People Search project - raw stream of consciousness, in bullet-point form. I didn't want to spend much time prettying it up, but reading the list of limitations, recommended customizations and preferred AD setup can save you weeks (and pain!) on your People Search project.

Cruft

• My SharePoint search page - to be clear, this is a static HTML page I made with search boxes to to search Google, search USENET, search the Technet forums, and search Google Reader. It's mostly broken now, and eventually I'll take it down. I used it A LOT while doing farm architecture-y type work, and used it heavily when troubleshooting in the early days. Now that I'm more development-focused, I've found I don't use it. Ever. Takeaway for everybody: Technet forums search covers more than Google does. If you're desperate enough, search both Google and the Technet forums (called MSDN Social now?).
• SharePoint search page - hottest of the hot! - wherein I add hotkey support to my (now-defunct) search page.

Op-ed (opinion pieces with almost no useful, actionable content—sorry)

• Dear MSFT, please talk to your Office division - op-ed, Sorry. Summary is, please don't obfuscate all your DLLs. Side note: InfoPath is pain.
• One Language A Year - wherein I dedicate a year to learn C#—that is, actually learn C#. I'll dig into Scala/Clojure/Haskell/Ruby/Python/Lisp/Scheme/Erlang/JavaScript/Io/Factor/OMeta/Smalltalk some other day. Also, I outright deny the claim that you should learn one language a year. It's cheap to give advice. It's not as cheap to follow advice. I have a new rule on following advice: does the person giving the advice actually do what they say? I got similarly disgruntled when "Uncle Bob" said something to the effect that you should dedicate 40 hours to work and 20 hours to learning. That's just crazy talk.
• SharePoint wikis are awesome, I swear - another of my top 3 visited SharePoint pages. I now apologize for defending SharePoint 2007 wikis. Afternote: I wish this wasn't such a popular page. Of all things, a wiki op-ed piece is one of my top pages, ugh.
• SharePoint: not unit testing - I've waffled a bit on this one. My current stance is that I'd really like to do continuous, automated functional testing (i.e. drive a browser window with code) to give me confidence my SharePoint solution actually works. True unit testing wouldn't cover enough space to give me confidence in my project, and most of my SharePoint projects are tiny, such that the "designing your API via design-by-example TDD" argument for TDD doesn't apply. Also, read this post for a short anecdotal survey on what kind of problems I run into when developing SharePoint solutions.
• Say no to makecab.exe - Here I rant against using makecab. I think I had just read yet another MSDN article that made casual use of MAKECAB.EXE and pretended like it was a good idea. Also I apparently just read the CodingHorror post on "Strong opinions, loosely held" which I now think is a terrible formula for my blog posts. At least I include a somewhat-useful PowerShell snippet that bypasses makecab, that's something.
• Surviving your first SharePoint project: Part 1 - wherein I sloppily argue that WSPBuilder is superior to STSDEV, VSeWSS, and makecab. It's true though, and somebody's got to counteract all these MSDN articles and books that pretend WSPBuilder doesn't exist…
• Does this describe you? - short, unhelpful post that quotes Niklaus Virth and laments SharePoint's accidental complexity.

SharePoint 2010/SharePoint 14 predictions

• SharePoint 14: Everything we know - it turns out from what I've heard from SPC09, this was dead-on accurate. They kept PowerPivot silent through the NDA period. Interestingly, SPC09 was silent on "Bulldog", the MDM product Microsoft purchased. Also I apparently missed out on the TownSquare bits, which they publicly discussed, and which evolved into the Facebook-like features.
• Preparing yourself for SharePoint 14 - I'm proud of my track record here, because I nailed pretty much everything. Written a full year ago.

"Other" category

• Yet another SharePoint VM: RIP - there was a period of time where I was Doing Something Wrong with my VMs. I now blame either/any of: a) saving state/restoring from saved state in Virtual Server and Virtual PC, b) running my external USB hard drive off of laptop battery power, c) lots of plugging and unplugging of said USB hard drive. I haven't had a problem in a long while now. Takeaway: back up your VM every so often "offsite", just in case.
• ASP.NET MVC is a MAGIC FLYING CARPET - wow, it's been two full years since the announcement! Anyway, here I mention how SharePoint development feels like alchemy sometimes, and separately, how the SharePoint developer community doesn't seem to value the things I like about ASP.NET MVC. Posting this had the side effect of sending lots of poor souls to my blog from google searching on "how to create an ASP.NET MVC app inside SharePoint."
• SUGDC Conference 2008: Recap - wherein I give a similarly-huge recap of each session I attended. Also: layoffs drive big SharePoint adoption! So, get with the layoffs!
• SharePoint + ASP.NET MVC - wherein I troll for people searching for these keywords.