Monthly Archives: January 2013

Painless and easy way to JSON endpoint in Sitecore aka your own API for Sitecore in few easy steps

Welcome dear reader.

In this article I want to dive a bit into helping you decide (or at least share some thoughts) regarding quite common problem that we had on last Sitecore project and that was : there was a need to have some sort of API (yes, I am under influence of ASP.NET MVC WEB API 🙂 and I like it) or some kind of endpoint that we could access from browser (or perhaps some device in future) and use it to query data, best nicely packaged as JSON objects, ready to use. Not that big deal you say, Sitecore is WebForms ASP.NET app, we have support for exposing JSON, it is possible. Yes it is and we do have the support. But we can have a bit trouble with fact, we are using Sitecore (since it is a platform, with many things added in web.config file, etc.) and if you are lazy as me.

Sitecore from version 6.6 (as described here) supports MVC (at last) and this can help us, but I was in a situation when we didn’t decided to use MVC so introducing this change for just one thing (my API) only would be overkill and too much work. So I searched a bit and I found out that the best solution for me would be a dedicated Web Service .asmx file (oh my, I took part on bigger MVC project before and going back to files from MVC point of view just doesn’t seems right).

But there are more options, why this asmx way. Let me elaborate more on them, but please first read this GREAT article about same problem but from ASP.NET point of view :

http://johnnycode.com/2012/07/16/getting-json-from-asp-net-beautiful-good-bad-ugly/

Done with the article?

Good, so here are my thoughts :

  • Web API would be the best solution here, if you look at how the code clean is and with fact in my mind – it is built for this very thing you need right now, but sry, no MVC this time, so there also goes my evil plan for dedicated MVC controller, well… But I would advice you go this way, it has its benefits,
  • Dedicated handler? Would work, but too much low level work (setting content type??? in 2013?? now come on),
  • Static method on some dedicated page (it works, I used this approach as a “service API” for one big Web Form but that was 3y back) would also do the work, but its really not elegant and since Sitecore has something like a routing (you don’t really access files in their physical folders, but traverse the logical tree in DB more or less), I had feeling it is not the right solution to my problem,
  • So here we are with my dedicated Web Service , some .asmx file that would do the job. IMO this was just what I needed, adding one dedicated file as a endpoint we can call will not ruin anything (ehm, there will be more files as I will show you later), we have support to just return something that will ASP.NET runtime serialize to JSON without much work (I will show you some tweaks I found on the web to optimize load and more), perfect fit for my needs.

Sitecore Item Web API 1.0.0

I have to interrupt this story to make it more complete here. Before I investigated how to send some items down the drain as JSON I had a talk with guys from Sitecore about what to use, they also said that there is one thing I could use besides Web API and that is their own product : Sitecore Item Web API, that was and still is in version 1.0.0 with date 28.08.2012. This is a good thing but has it’s weaknesses (but maybe it was just me, although I have studied also documentation but maybe you can prove me wrong and this tool is what I needed) and strengths. More on them here :

  • You can (and should) use Advanced Security where you explicitly have to tell what fields you are exposing and then either query particular ones or don’t specify any and you should get all,
  • I wasn’t able to specify what branches of my Sitecore can I query so I was able to query everything. Also system parts of it. This I really regard as a security issue (or it was just me and my mistake with setting things up),
  • Not needed data were sent down the wire like ID of record, whether is has children, what database I am accessing and more. Stuff that just wasn’t needed and again was a security issue (from my point of view),
  • Desired properties were stuffed to one property of item and if property had space in the name, I had to use this approach : http://stackoverflow.com/questions/4042646/how-to-access-json-object-which-has-space-in-its-name,
  • Regarding using dedicated account with own name and pwd, that might me the way for some “hidden” app that would need API access via web, but if I would write JS code in some lib and use it, that would be the first thing where I would look if I would like to try to hack my app,
  • The final thing I had problem with was XPath query with some parameter that I had to URL encode to get it work and it worked, but then suddenly a business requirement came that would need me to query API, assemble some information, do something with it and re-query for final results. But this is business logic that isn’t supposed to be on the client (another example could be some simple JS validation logic that would make sense, but not this type + one more HTTP request = more possible delays, more hits to DB…. that just wasn’t good)

I just wasn’t fully in the control of this thing. So I decided to investigate on possibility that give enable me full control of what I send down the wire. And encapsulate some business logic that could be reused somewhere else, in code behind, etc… Client doesn’t and should have to know about relationships between objects, how I can get what from this and that. APIs are the encapsulation of some business logic, in this particular case exposing it to HTTP access via Web Service. Or at least this is my understanding of how things should work.

OK now some good advices I learned while working on the project, these are more or less generic to ASP.NET as such.

JS code

JS code could look like this :


$('#value-button').click(function () {
$.ajax({
type: 'POST',
url: '/Service/Service/CustomAPI.asmx/ComputeMeValue',
data: '{ "value" : "' + $('#value-input').val() + '" }',
//data: JSON.stringify(data), - use this if you have custom json object with properties...
contentType: 'application/json; charset=utf-8',
dataType: 'json',
success: function (data) {
$('#outcome-paragraph').text(data.d);
},
error: function (data) {
alert("something bad happened, we are sorry...");
}
});
});

This is a small snippet from sample project I will post with this article.

I would advice you to have also error part in the JS code so once your code throws some exception, user will be notified.

If you are not posting parameters, you should still send empty {} in JS code, or send values you want on the server. Names and types have to match obviously.

Web Service code

At first, there are more approaches here. You can layer/separate you code in own project if it gets big, but for my small API I was able to get away with subdirectory for Model, where I stored POCO classes (in MVC project this could go to separate Class Library) and with separate subdirectory for service as such.

The service could be divided to

  • BLL layer that could store business logic (how to combine objects, how to create VMs – view models, what to query for what),
  • Mapping between domain objects (object we have from Sitecore API, or EF or some ORM mapper in case of MVC) and VMs we have in Model subdirectory (this could be in BLL or as separate library, or in MVC in the model project, if you don’t have BLL project),
  • MVs – View Model classes in Model subdirectory.

We could also use DI – dependency injection as we do in MVC but all this and above really depends on, if it makes sense for you or not, if it helps you or its overkill on smaller project. If you will/need unit test or not. Your project, your rules. (but unit tests are good, for you :))

Small tweaks to the code (the price to pay for Web Service .asmx approach)

Declare the type of what you return as object (I found out this on the web after few hours of pain with dynamic, that I was used to from MVC….). This has some nice side effects. First of all you can return either string, or int if something goes wrong with the query and there is nothing to send back, if you have some data to send, then just send List<User> for example. Neat thing to save bandwidth is that you can send down objects of say User and SuperUser and AdminUser that can be extended with some new additional properties and these will be just on the items that have them. You have to declare the object that will be stuffed with these different objects as List<object> and then you can add to it what you want/need. After you are finished, just return it. But you have to deal with this (different types of objects) in your JS code of course. The last thing why I would advice you to use object as return type is, that plumbing in .asmx file will send down in JSON for each object also its type, like this :

web service json sample

Just by declaring return type as object, with the very same code you will have this response :

web service json sample with object return type

Magical, right? The best time to switch to ASP.NET MVC 🙂

Another thing I wasn’t able to make work was something like ASP.NET MVC model binding. MVC has nice functionality that will turn parameters from cookies, query string, posted values, all to one object that action on controller has as parameter (matching names of values and properties on object). So you don’t end up with 30 parameters per one action. Here you have to use older approach with no binding, I am sorry.

OK so hopefully I have not forgot something, if you have any questions, just write to the comments, and here is the promissed link to small sample page for download : http://sdrv.ms/WZmVq5 enjoy.

How to free up and retain free space on lowend older Android devices with small memory

Hi all owners of older Android phones, I belong to you and we all share the same problem, low disk space….

One year back I bought myself a ZTE Blade I, device with Android 2.2 (I had mine with 2.2 at least), +-150MB of space in the device. It’s low cost device with good display (device is actually about 2-3y on the markert and it came with 800×480 display , what was at that time and for that amount of money quite a shock). A perfect fit for my needs at that time.

Anyway there are more older phones or Android devices that have same problem (like HTC Desire, etc), install some apps and there is no room to install more and although you moved to SD card what you could, phone still has good battery, good display and is not worn out, you just can’t use/enjoy it.

So here are my few points what you might do :

First of all, grab this app : DiskUsage :

https://play.google.com/store/apps/details?id=com.google.android.diskusage , from Ivan Volosyuk, thank you mate for this one.

I don’t want to duplicate how app works, it’s all on the Google Play sub page, but I would like to share some things I discovered on my device with help of this tool (I used it to scan my internal memory and identify apps that take most space. It’s really easy.) :

  • All the updates of Google’s own apps like Voice search, Street view, etc, all these don’t just update (overwrite) older versions, they install in some reversible state, so you have on your precious small drive apk from previous version + updated one and you can revert back (not sure if this can be changed in any way, please share this information if you have it) . My advice here is to remove all updates on Google apps you don’t use. Or update once needed (anyway at least Google Play and Gmail will be updated since I suppose these are integral part of your phone and you do use them, in other words, it will add value to you when you update),
  • Biggest hogs I found on my device were browser with 20MBs!!!!! of cached data. I was erasing the cache from browser’s menu, but this freed only +- 4-5 MBs. Once I removed this, all my cookies and my stored links went down the drain, but whatever. Another hog I found was gallery with +-4MBs of some cached things and Google Play store app as such, which BTW installed some I suppose “deamon” for downloading the apps once you click on them on page. It takes up 7,5MBs and when you try to uninstall it, it will install back in the background,
  • I don’t want to repeat thing that are everywhere on the internet like move apps to card, or some other tweaks (root and/or download SDK and set default install location to SD card… etc.), you already know them and use them.

Hope this helps some of you and it will extend life of your devices at least a bit 🙂 You can always use them later for geocaching of as sport tracking device as I do 🙂

Cheers.

Facebook, Google Drive, SkyDrive how well is our data secured?

At first, the thing I will write today about in short is not something new that I and only I came up with (since I am not some security research dude, I am just a common .NET dev), its something that has a lot of articles about (and its funny that some articles are perhaps 3 years old) :

http://www.theaustralian.com.au/australian-it/facebook-images-open-to-access/story-e6frgakx-1226059138255

http://www.securityninja.co.uk/hacking/access-any-album-on-any-facebook-profile/

http://allfacebook.com/facebook-photos-warning_b4065

http://www.niallkennedy.com/blog/2009/04/facebook-haystack.html

http://arstechnica.com/business/2012/08/facebook-finally-changes-photo-deletion-policy-after-3-years-of-reporting/

and tech savvier users are using a lot (without even realizing) and that is :

copy over URL of image from lets say G+ and posting it over to FB, or getting the URL of some resource as such.

Small example:

Go to your G+ feed, flick on an image, right click image and click on Copy image location (this time in FireFox) :

copy image location firefox

So you have URL like this (this is the image above): https://lh3.googleusercontent.com/-FkqW7ThbMcc/UP0Y7RRyTgI/AAAAAAAAUtA/KElCS3OUSIY/s800/599597_10151267147710219_2139233647_n.jpg

Its long and cryptic but its a URL and once you have it, you have the absolute position of image in Google’s cloud storage (btw this is how it works in general, also for Facebook CDN).

Its logical, files have to be stored somewhere, so you can access them (download them). But here comes trouble:

All of these big players are using CDNs – Content Delivery Networks. Using CDN is design decision and for each decision there is a price to pay. CDNs are dedicated machines that have only one purpose : serve content and serve it fast. You can picture them as warehouse stuffed with data. But without a map (or at least no public map, but there are some servers that index some type of URLs everywhere, like rapidshare files, etc.). When you have precise address of what you need, they will give this to you and since they are optimized for speed, data is cached and no cookies are sent back and forth so user in fact can’t be recognized and NO security decision is made whatsoever about if CDN should give you the content or not (this is true for solution from FB and Google, not for Microsoft, I will show you later). The logic behind if you should see some posts or content is made elsewhere, in the layer that sends you the HTML for your browser. You decide, to whom should these URLs will be revealed, BUT once someone will copy these and give them to someone else, then you are in trouble, he can access them and as I found out on Facebook, my image is there also after I deleted it, will see how long that will take to remove it completely. (and yes, of course even when this wouldn’t apply, someone could just copy images to his HDD and misuse them in any way).

So this is what I have found out about particular pages. I have chosen to dig deeper on 3 of those, on Facebook, Google Drive and on SkyDrive. I did my test with image, since I can only upload image to Facebook’s CDN so far and my investigation started with images and their security. (One might disagree on sites that I have chosen to compare, but I just want to elaborate more on security of place where the actual content is stored), I don’t want to harm any of these pages and I am also no trying to convince you about which one to use. However I personally understand why Facebook made his choice, but I don’t understand why Google Drive doesn’t have better security checks, since I don’t think there will be big load on files you want to share with colleagues, etc unless you are some celebrity or something.

Facebook:

Facebook stores its data on URLs like these ones :

https://fbcdn-sphotos-e-a.akamaihd.net/hphotos-ak-ash3/601115_10200262915483090_1662049204_n.jpg

There are no cookies (extra load) transferred with the request and there is no security check on these URLs, you are free to send them to whoever you see fit

fb cdn request

  • You even doesn’t have to be logged into Facebook to see content, since this is different domain then Facebook.
  • No security rules apply to these links, try to set it to only me, link still works for everybody.

How to get to the URL of image: just right click it and “Copy image location”.

Deletion of image: even I have removed my image, its still there on the CDN, try this link (image was uploaded as private, then deleted today 21.1.2013 +-10CET):

https://fbcdn-sphotos-e-a.akamaihd.net/hphotos-ak-snc7/304916_10200271924588312_192680811_n.jpg

However one of the posts in the beginning of this article says about some timeout for deletion of images, I will update the article once image will be removed.

Google Drive:

Google stores files on URLs like this one :

https://lh3.googleusercontent.com/-EsRO6HElcuI/UP0P9-wqmPI/AAAAAAAAFws/gJFzDVOr_QY/s400/run_all_unittests.jpg

Google drive and Google Plus and as far as I know Gmail, they all use same CDN to store our files. So what I did was, I uploaded image only for me and investigated on URL. Google made a small maneuver to trick user and that is, there is a div above the image with class name : tm-image-glass. So when you right click on an image, you don’t have “Copy image URL”. Anyway the don’t have this on Google Plus (you can right click and have the URL instantly).

Here you can see the request :

google cdn request

Same thing, file is served from URL I have found in Google Drive page, no cookies, no security, no restrictions. You don’t have to be logged to Google account. Why this decision, hard to say.

Deletion of image: Once the image is moved to bin, user will be replied with 403 error page, which is desired behavior. After I restored it, it was back online.

How to get to the image URL: little bit harder, you have to ding into the HTML like this :

google cdn url found

Microsoft SkyDrive:

Microsoft is also using dedicated CDN machines and image URL looks like this one :

https://byfiles.storage.live.com/y1p41KbXCXgw1n-4inHc4Vs2gL7D5lHjUf0YmamC5eVVdtPA6BGoQvX7A/run_all_unittests.jpg

SkyDrive the answer from Microsoft to all these cloud storage apps that came recently (same as Drive from Google). The intention is not to host just images, but also documents, etc. SkyDrive is checking who are you, when you want to access the image and will refuse give it to you, when you are not logged in with credentials of user, you gave permission to (I tried to log in with different account and that was just not enough).

However there is a price to pay:

skydrive cdn request

And that is : cookies, cookies, lots of cookies, cookies everywhere!

Cookie monster would be happy.

I have also encountered some things that maybe shouldn’t be there, like X-MSNServer name :

skydrive cdn request headers

or some 404s:

skydrive cdn request 404s

That really shouldn’t happen in production guys.

Deletion of image: when I removed the image, I had 404 from the server, after I restored it, it had same URL and I was able to access it.

How to get to the image URL: just dig deep in the HTML, its there:

skydrive cdn url found

There is no native right click menu of the browser, there is only customized menu from SkyDrive.

So the outcome/state, as I see it right now (21.1.2013) :

Facebook and Google Drive are using fast unsecured URLs, SkyDrive slower but secured (the are checking the cookies). The state might change, anyone can upgrade their systems and I hope that will happen sooner than later.

If you have any other findings, more information or you think I have missed some spot there, please feel free to comment or extend this article in any way. Thank you.

How to empty cache of Microsoft Windows 8 Photo application (my hacky approach)

At first you might ask, why one would like to do this kind of thing and why even write a whole damn article about that. Well the answer is not so common but simple.

Picture a situation where you need to return notebook or PC or any device with Windows 8 installed. I have Windows 8 on my dev box since beta and when I received notebook at work, I have chosen to link my account on the machine with one of my live ids (or Microsoft account as they call it now). I wanted to see how good is the new Photo application so I linked my personal Facebook and it downloaded a bunch of photos. So far so good. Now I need to return the notebook and I wanted to unlink my Microsoft account. I had luck to quite easily find how to do this in Store, in Mail and People app but though I unlinked my account from photo app, pictures were still there, cached somewhere in my HDD. Damn, now what? No clear, no empty, no remove everything option…. :/

Just to speed you up, here is what you can do and what I find the best and easiest to do, then I will dig a bit into the second possible scenario.

  • create new user from the scratch in Control Panel\All Control Panel Items\User Accounts\Manage Accounts (say admin, something, just that the machine has some root account when you return it back and then erase your personal account from that new root account), but this is the way you loose all your data associated with that account on this particular machine
  • you are not giving the machine back, just want to switch to another Microsoft account, so please read on, in short, you can uninstall the Photo application, remove directory where it is storing its stuff and then install the whole app back

(just a small side note – I have tested them both after it hit me, that new user is also a way to go and it will give you really clean account)

The first option is the cleanest, since Windows 8 apps are living in this folder :

C:\Users\your_account_name\AppData\Local\Packages

You can quite easily hack there and sniff what they are doing. The folder that I had to find was this one :

C:\Users\dusan\AppData\Local\Packages\microsoft.windowsphotos_8wekyb3d8bbwe

This is the folder where all my photos were stored. First I tried to removed all .jpg files. Just a partial success, app was working, but it was able to somehow retrieve its pictures back. Part of them. There is one big file, I suppose its some kind of local DB, but didn’t had the time to investigate on that more. So I just removed all of the files. Rerun the app – bam – app didn’t even started… Good job guys… Now what? OK, I tried to uninstall the app (with empty folder), you can easily right click on the tile and uninstall it. Then I installed it back and its in the default state… So that was my path to those two bullets that I would advice you to choose from, when you are in the need of unlinking your Microsoft account from Windows 8 for any reason.

But maybe I am just wrong and overlooked some option somewhere, so please if you have any better/faster/this is the way it was supposed to be done, share it please in the comments.

Thank you and enjoy.

Fixing the build errors in Visual Studio after inserting new sublayout to Sitecore via backend web page

This short blog post is about well know fact and all Sitecore veterans will just smile, but for all newbies like me, it will save you some time.

So, after you insert new sublayout or master page, there is one obvious thing you should do and that is :

  • include the file to project in Visual Studio, that you can work with it.

Then you will build the solution and you are happy, all works as expected. But here comes trouble. There is something missing, one file that Sitecore is not creating by itself and that is :

  • .designer.cs file

Then you might to have some error like this one, but only after you add some control/controls :

sitecore convert to web applicaton error

Basically, this file is needed for VS 2010 and VS 2012 to be able to work with objects you create in .aspx file also in code behind file. VS is creating .designer.cs file and maintaining them by itself, but Sitecore is not. You have to do this by hand, like this :

sitecore convert to web applicaton

You can do this over whole project, or just over new file.

Hope this saved you some time.

Some more articles about converting to Web Application Project :

http://msdn.microsoft.com/en-us/library/aa983476%28v=vs.100%29.aspx

http://stackoverflow.com/questions/735054/how-to-convert-asp-net-website-to-asp-net-web-application