I'm not sure if I agree with your sentiment Paul. Generally, we've seen HTA mode as being much more stable than IE PI mode or even IE multi-window mode. I suggest we let Dan and Philippe (who are both on vacation but will be back next week) come back and justify their decision.
Generally, I thought the conclusion from our last meetup was that for a 1.0 final release we would change the default modes to use the recommended browser launcher and that we weren't going to enforce API or execution compatibility until we hit a 1.0 final release.
Patrick
So, let me clarify one thing -- *iexploreproxy uses exactly what *iexplore
used to use, right? (I can check the source in a little bit). That is one
of our potential solutions, namely because finding all of the occurrences of
the use of the location properties is 1. tricky and 2. there are a lot of
them.
I think it does. I can't recall if *iexplore used PI mode or not. It's gone through a few changes, so depending on which build you have the answer may be different.
We aren't finding any other significant 'bugs', per se. It's mainly that
many of the tests running here use these fields, which were legal in
*iexplore, and are no longer legal.
Can you provide us a test case that reproduces this issue? I think that should be the first step to any action we take (including rolling back the recent change).
Out of curiousity -- why call the new *iexplore *iexploreproxy if it doesn't
use proxy injection?
Yeah, it sounds confusing, but there is a reason
PI mode is where Selenium is actually injected in (via a proxy). But the old "original" mode also depends on a proxy as well, since it needs the proxy to make the Selenium browserbot code appear on every domain.
We always start the server with -multiWindow, and the calls to selenium which opened up IE were using *iexplore.
OK, good. Generally our feeling has been that HTA is better than other modes, but that "original" mode + multi-window was better than just "original" mode by itself. So if HTA is giving you issues, then you've got the next best option.
Now, *iexplore was changed to *iexploreproxy.
Yes, I think, though you might want to double check to make sure your build wasn't one of the ones that had iexplore mode actually using proxy injection (I doubt it, but can't hurt to double check).
At Google, about 30% of the tests that we run have some integration with code that will not work with the HTA browser because of the permission denied issue. This will require us to go back to using full blown IE and changing all the targets to *iexploreproxy.
My suggestion is to change the default mode back to full blown IE and then make HTA mode available with some other target name *iexplore-hta perhaps
Any reason you can't change your launch strings to just use *iexploreproxy? Or, if that's hard because it's embedded in the tests, you could easily overwrite the iexplore browser launcher for your Selenium Server instances that sets iexplore back to what you need. Check out BrowserLauncherFactory.addBrowserLauncher. I bet that will give you the quickest path to victory.
Source code here:
Erik's previous link to the Microsoft web site shows the incompatibility
that some of our tests have with HTA mode - certain tests are using location
properties that are not accessible in HTA mode. These tests will never be
able to use HTA until they are changed, which, given their quantity, will be
quite a while from now.
You bring up an interesting alternative with the addBrowserLauncher. We
will look into it.
I look forward to Dan and Phillipe returning, but here's the part I don't
get. You have a Map, with two distinct keys pointing to the same value,
with that being the case because you want to drive users towards a more
stable model. I understand the goal behind this, and I understand the
sentiment of not maintaining backwards compatibility with respect to the API
of the clients (signature/parameter changes, etc.). However -- given that
you have existing users, many of whom require the original *iexplore
behavior because of their use of javascript identifiers that Microsoft,
not you, have said are inaccessible in that mode, making this particular
change go beyond the API compatibility question.
If *iehta mode were compatible with javascript in *iexplore, I would not
have a problem with this. We'd accept the change and eventually, without
such an immediate need, modify our launchers to say "*iehta" for better
readability and understandability in what was being launched. That is not
the case, though. They are not compatible, and by changing the mapping we
force users to make the decision to either upgrade RC or change user-side
code. This is not an easy choice for us: we want to upgrade, rather
emphatically, but the change required is significant. Not all of our
Selenium testers use the Selenium Farm here. If that were the case, then
either of the two methods proposed as a workaround would be much easier to
manage, but as it is we are guaranteed to cause some tests to fail during
the transition on the Farm, with a much longer tail of failing tests as
other users have to stop what they are doing and make the same change.
I agree with Paul in that if you want new users to have the easiest and
smoothest experience with Selenium and IE, that the documentation say that
*iehta is preferable to *iexplore, much as and for similar reasons as
*chrome is preferable to *firefox. Specifically call out the three location
properties Erik mentions as being an incompatibility for *iehta. Then let
the users pick what they need to use. Using dual distinct keys to map to
the same value decreases understandability as well, because given a launcher
"*iehta", and "iexplore", I'd only have expected the first one to launch in
HTA mode.
I don't think we necessarily should remove the entire code branch from the
change that made the mapping - as I understand it, several of those were
fixes that would be a good idea anyway.
We don't want to have an internal version of Selenium used on the Farm.
We've tried that, back when we first started using it, and merging was a
royal pain in the rear. We'd much prefer to use a vanilla version, because
that way, our users who do not use the Farm can write their tests and then
have them just work when they are automated to the Farm.
I suppose, the gist of this email is that the change where *iexplore
launches IE in HTA mode is beyond the normal considerations of 'backwards
compatibility', because it is not the rest of the Selenium code that breaks
compatibility -- it's a basic, low-level property of HTA mode itself that is
incompatible. I've upgraded Selenium several times prior to this, changes
which included signature changes that we could not hide using reflection,
and none of those were as difficult as this upgrade is going to be,
regardless of which of the two options presented are used.
Thanks for listening, /rant.
-Jen
Here is some more information on the problem of changing the default browser target to HTA mode instead of leaving it as the regular Internet Explorer. Take a web page as simple as the following:
<html>
<script type="text/javascript">
function applicationValidate() { return window.location.href; }
</script>
</html>
It can do any number of operations but the key is that the web application under test uses the window.location.href object.
Then if you have a selenium test that runs the following
String script = "var win = selenium.browserbot.getCurrentWindow(); win.applicationValidate()";
String pageTitle = selenium.getEval(script);
You will get a permission denied error now, as of revision 2450 because of the change from the default iexplore target to the other hta mode target.
I believe that a lot of users who upgrade from the older version of selenium to 1.0beta will experience issues with their test suites or test frameworks, especially if they are trying to test javascript on the web application under test. The fact that rev 2450 causes this HTA browser limitation to be exposed as a default way of running selenium is extremely discouraging for people who want to upgrade.
Erik
I stand by our decision to make HTA mode the default mode.
First, to answer a question raised earlier: why's it called *iexploreproxy, when it isn't using proxy injection? There are actually two "proxy" modes available for IE. In Proxy Injection mode (*piiexplore), every HTML page is modified to include Selenium. In the other mode (*iexploreproxy), the proxy never modifies pages. Instead, it makes a phantom copy of Selenium available on every domain. It's an alternate way to defeat the same-origin policy.
In *iexploreproxy mode we can test only one domain at a time, but it can be any domain. If we want to test example.com, we load up a Selenium frame at http://example.com/selenium-server/. Selenium scripts on example.com can be used to read/modify any page on example.com, so they appear to work as normal. But we're screwed if we need to test multiple domains, even if the domains are very similar. For example, we can't test both "www.example.com" and "example.com" in the same test. Nor can we test "www2.example.com", nor can we test both HTTP and HTTPS pages from example.com in the same test.
In a very normal use case, this means that the user can't login to their web application, because logging in switches the site from HTTP to HTTPS and then back again.
The limitations inherent in *iexploreproxy are very severe and affect many ordinary users of Selenium, especially those who originally wrote their tests in Selenium IDE where they aren't confined by origin. Today, these users reliably come to the forums to ask why their test script works in Selenium IDE but not in Selenium RC; it often takes quite a bit of diagnosis to find out that they're crossing the domain boundary.
There are no inherent restrictions in *iehta mode, despite the linked MSDN kb article. Those restrictions apply only to the HTA frame, to Selenium Core itself, and not to the application under test, which is run in a separate window by default. That window is free to access window.location and do whatever it needs to do with it. Selenium has to be written very carefully to handle this case, but ordinary users will not and should not be affected.
As I'm sure you're aware, there is no one mode that is best for everyone; I'm well aware of the limitations inherent in HTA mode, not least because of the numerous workarounds I've coded in Selenium Core to support HTA mode's quirks. *iexploreproxy is better than *iehta when it works at all, but if you need multi-domain support, as most users do (whether or not they know it), you simply cannot use *iexploreproxy mode at all.
-Dan
Erik, your example is interesting, but we can fix it (as we have fixed numerous other Selenium Core bugs). getEval currently runs a raw eval in the context of the HTA frame, but we could make it do the eval in the context of the application under test instead. (We'd have to get clever with timeouts, but it's not insurmountable.)
With that said, there's also a straightforward workaround available today: you can use runScript instead of getEval. runScript launches the script as an inline script tag in the AUT, where it runs under the AUT's security context. No HTA restrictions there. Alternately, there's an even easier workaround available to you because you're running a single-origin test: just use *iexploreproxy!
I've fixed a great many of these HTA bugs. There are a finite number of them; they're pretty easy to fix once you understand them. I'd even go as far as to say that we've quashed most of them. Reading this post doesn't really worry me; it just encourages me to fix the bug.
-Dan
Hi all,
I'm glad to hear that you think that this can be solved by a bug fix. I
would like to clarify one thing.
We do not have a problem with HTA being the "default" mode. Our issue was
in how "default" seems to have been implemented, because our tests are no
longer getting what we ask for. The Farm actually does handle tests that
want HTA mode via the *iehta mapping. Because Googlers were such quick
adopters over a year ago, however, HTA mode was not stable enough at that
time, so many of them worked within the constraints and limitations of
*iexplore.
Thank you for the workaround suggestion, but we cannot upgrade when it
requires changing the tests: every one of our Selenium-using projects has a
different schedule and requirements and cannot be retasked to do a code
scrub for the purposes of upgrading. We similarly are unwilling to, on the
fly, replace "getEval" with "runScript", for the same reason as above: we
must give people what they ask for.
That said: will we encourage our Selenium users to change their tests to
make them better able to take advantage of improvements in Selenium? Of
course. What we cannot do is impose when they do this.
One more clarification: the *iexploreproxy change is feasible for users of
the Selenium Farm, although it will not go unnoticed by the continuous
integration systems. The reason that is not a simple workaround is because
Google also has many Selenium testers who do not use the Farm, and each and
every project in that category will have to make a similar change.
The only "easy" workaround we actually have is the one where we replace the
mapping of *iexplore to the original browser launcher, either by modifying
the code or surrounding the vanilla RC jar with one that includes the
wrapper that Pat suggested. I will strongly regret making either of these
choices, because they represent an inability to use "off the shelf"
Selenium.
Again, I am glad to hear that you think that the primary means of
incompatibility that we have found with the *iexplore -> HTA mapping is
something that can be fixed in Core. When do you think that would be able
to be done? If it is not in the next week, we will have to go with a
branched version until it is in place.
While it's true that I may still "grumble" about not getting what I asked
for with *iexplore, if there are truly no incompatibilities between HTA and
pre-2450 modes, then I will certainly keep it to myself :).
-Jen
I personally don't have an objection to making *iehta the "default" mode. What I find somewhat problematic is the renaming of an existing mode to be something completely different. People's brains, which were previously wired to understand *iexplore != *iehta now have to undo that association.
Might we not simply introduce a new *iedefault launcher as a synonym for *iehta, and simply reference *iedefault as the "preferred" way to get started with Selenium in the docs? That way we could leave *iexplore as is, and potentially save a lot of people from having to update their tests.
Haw-Bin
Message was edited by: Haw-Bin Chai
Yes - Thank you, Haw-Bin, for more clearly stating my concerns :).
-Jen
IMO, long term (maybe short-term?) I'd rather that we implement mode switches with launcher options, rather than as separately named launchers.
We shouldn't have *iexplore, *piiexplore, *iexploreproxy and *iehta (and *iedefault?); instead, we should be doing something like this:
cmd=getNewBrowserSession&1=*iexplore&2=http://google.com&3=mode:hta
cmd=getNewBrowserSession&1=*iexplore&2=http://google.com&3=mode:proxyInjection
cmd=getNewBrowserSession&1=*iexplore&2=http://google.com&3=mode:singleOrigin
and, when launching firefox,
cmd=getNewBrowserSession&1=*firefox&2=http://google.com&3=mode:chrome
This doesn't entirely solve the problem; what should we do if you don't specify a mode? But at least all Google employees could start saying "mode:singleOrigin" today, even with Selenium 0.9. (0.9 will discard unrecognized options.) When upgrading to 1.0, unrecognized preferences would become activated.
As for "*iedefault", I'm not sure how that would translate into launcher options, but I'm not sure we'll need it if we implement a launcher options scheme, right?
-Dan
Despite my flip remarks earlier, fixing SEL-587 the way I described would have a drawback: you could no longer write an eval with elevated privileges... you'd HAVE to be within the application's reduced security context. That seems bad; I'm not sure how bad or what I think about it. I really don't want to make this any more complicated than it needs to be...
This mode-based approach would work for Google. It would give us the
ability to set an internal Google default to singleOrigin right now, as you
say, and the "*iexplore" mapping would continue to work as prior to the
mapping change. Most if not all project teams would then be able to avoid
test breakages until their schedules allowed them to take a look at which
tests could run in other modes.
-Jen
True. My main goal with this set of discussions is to find a means whereby
we can upgrade without imposing breakages on the teams that don't use the
Selenium Farm. The mode-based approach ought to do that. With a smooth
upgrade path possible, I don't see as much of a win in trying to provably
eliminate all incompatibilities between HTA mode and the singleOrigin mode..
-Jen
