>
New Topic
>
Reply<
Esato Forum Index
>
Sony Ericsson / Sony >
General
> SE ranks poorly in reliability/quality
Bookmark topic
Quote:
On 2005-08-31 14:01:07, max_wedge wrote:
In the Which survey, 20% of SE users reported faults, and 10% of Nokia users reported faults. The survey sample was 50 users, hardly enough to make any sort of realistic differentation between a 10-20% result.
Seriously, this "survey" would not be considered a reliable sample by any professional statistician.....
Everybody read this report and decide for yourself if the method was accurate or not. I myself am appalled it even made the news. It's sensationalism as far as I am concerned....
http://news.bbc.co.uk/2/hi/technology/4745205.stm
[ This Message was edited by: max_wedge on 2005-08-31 13:04 ]
Pls dont jump the gun. I think sample size was
5000+ not 50. The 50 were related to phones specifically used in 3 network. I think BBC (or anyone in their right mind) would not pick up the report if it were 50 sample size.
[ This Message was edited by: mib1800 on 2005-08-31 13:38 ]
[ This Message was edited by: mib1800 on 2005-08-31 13:49 ]
--
Posted: 2005-08-31 14:36:13
Edit :
Quote
oops!
Okay fair nuff. That's not so bad. However, I still don't think 10% difference is that big. In two months time the result could be the other way around. The mobile phone market is very fickle.
Besides the survey does say that there was no differentation between network and handset faults, so I still don't think the survey is truly representative of manufacturers. After allowing for which are hadnset, and which are operator, it might turn out that SE have been unfairly branded (no pun intended
) by a poor operator performance.
I still think the survey needs to be better designed to really see the true trend.
--
Posted: 2005-08-31 14:58:11
Edit :
Quote
I think alot of damage was caused by the recent K750i launch / freezing issue, which I think has been repaired now.
--
Posted: 2005-08-31 15:04:39
Edit :
Quote
@max_wedge:
if the 5000+ is statistically representative, the 10% difference can mean hundred of thousands (or even millions).
Like I said before, it doesnt matter whether it is network or phone faults. I think the survey is more phone-users oriented than manufacturer. What can be concluded from the survey is that those who bought SE/Moto are twice as likely to be inconvenienced due to the need to take the phone in for repair/service.
[ This Message was edited by: mib1800 on 2005-08-31 14:23 ]
--
Posted: 2005-08-31 15:21:17
Edit :
Quote
Quote:
On 2005-08-31 03:46:50, mib1800 wrote:
You can't expect others that carry the news feed to print the full article, do you? You cannot say it is meaningless just because you dont know the full details. It just like saying the quarter sales figure issue by SE is meaningless because the news feed did not carry the full breakdown details (running into tens of pages).
I can expect that, whether they do it is another matter.
The sales figures you mention would be somewhat meaningless because they only give one piece of information without further details. As I said before, such reports can suggest trends or display a certain reaction, but without aproper analysis and breakdown they aren't really all that useful in guaging the full situation. One can take something from them.
Quote:
Most of the time we are just interested in the final conclusion.
Which is a problem.
Quote:
Of course those who were given the bad limelight would try to rubbish it. btw 3 didnt provide alternative figures to disprove the findings except only saying sample size is not big enough.
I couldn't care less whether 3 provided figures or not, their point still stands. The sample size of their customers was very small, they don't need alternative figures to prove that.
[ This Message was edited by: mib1800 on 2005-08-31 03:14 ]
[ This Message was edited by: mib1800 on 2005-08-31 03:17 ]
[/quote]
--
Posted: 2005-09-01 00:27:09
Edit :
Quote
scotsboyuk:
Quote:The sales figures you mention would be somewhat meaningless because they only give one piece of information without further details. As I said before, such reports can suggest trends or display a certain reaction, but without aproper analysis and breakdown they aren't really all that useful in guaging the full situation. One can take something from them.
You are supposed to digest the INDICATIVE trends information and not do a micro analysis which will bring out another hundreds of other micro-factors. You are trying to disprove the big picture just because some of these micro-factors show contrary information. Have you ever thought when all these these micro-factors when brought together will give you the exact general conclusion/trends.
Whether you like it or not statistics are never exact science. It is just an exercise in futility if you try to make exact science out of it. All is does is give us some indication that something may be going right/wrong. At least this is much much better than "gut feel".
So can this survey be wrong? Yes, there can always be this possibility. And so does all the other surveys which accorded SE shining accolades.
.
Is the survey meaningless. Absolutely No. By reading this hopefully the person is wiser (at least he/she will pause and think before making silly rhetoric like "xxx makes the greatest phone in the world" or "yyy makes the worst phone in the world")
Quote:I couldn't care less whether 3 provided figures or not, their point still stands. The sample size of their customers was very small, they don't need alternative figures to prove that.
OK. So what do you think is the sample size of SE phones within the 5000+ respondents? Is it big enough for you? If you say yes, then I would tell you the same general conclusion from the survey as I've told max (in previous post). If you say no, then see the paragraph below.
Dilemma. Say someone wants to do a comparative survey/analysis of different brand of phones. Lets take a hypothetical figure of 5000 as a good sample size (i think you would agree with me on this number). But since SE market share is only 6%, in order to get minimum of 5000 for SE, you need to have about 80,000 respondents. You tell me which survey can get this kind of respondents short of a population census? So you are saying we can all dismissed all these comparative surveys as being meaningless. imo, you are opening the pandora box whereby anyone can rubbish any survey that awards accolades to a small player like SE.
[ This Message was edited by: mib1800 on 2005-09-01 03:19 ]
[ This Message was edited by: mib1800 on 2005-09-01 03:46 ]
--
Posted: 2005-09-01 04:16:55
Edit :
Quote
One SMALL teency sticking point..
Like I said before, it doesnt matter whether it is network or phone faults.
If you're going to call into question a manufacturers reliability then the above matters a great deal.
Lets say for arguments sake that everyone of the
respondants just happened to be an orange (it was orange wasn't it?) user with a K700i, the txting issue that was at fault within their network would have made an absolute joke of any statistical result.
After all, by that token CD/DVD burners would have horrible reputations.. Given that many are based on the cheapest shoddiest media people can find to use in them. It must however be the units fault, not the media used in it.
--
Posted: 2005-09-01 05:19:11
Edit :
Quote
Quote:
On 2005-09-01 04:16:55, mib1800 wrote:
You are supposed to digest the INDICATIVE trends information and not do a micro analysis which will bring out another hundreds of other micro-factors. You are trying to disprove the big picture just because some of these micro-factors show contrary information. Have you ever thought when all these these micro-factors when brought together will give you the exact general conclusion/trends.
Generalisations are not something I often appreciate. I prefer to consider all the facts and base my opinion upon them. A conclusion is only one's opinions based upon the underlying information, which may be subjective.
As I said before, this report indicates a trend, without the full list of parameters used and an indepth analysis of the methods employed I don't consider it to be of any significant value. I was trained as a scientist and as such I was taught to question and look for the underlying causes. Trends are one thing, what they are based upon is another.
Quote:
Whether you like it or not statistics are never exact science. It is just an exercise in futility if you try to make exact science out of it. All is does is give us some indication that something may be going right/wrong. At least this is much much better than "gut feel".
First of all you know very well that I am not a fan of employing statistics (which is why I wasn't best pleased to have write about them below). Secondly I am not trying to make an exact science out of it. Thirdly you agree with my point about trend indication.
Quote:
So can this survey be wrong? Yes, there can always be this possibility. And so does all the other surveys which accorded SE shining accolades.
.
Hence why people shouldn't put much stock in such surverys.
Quote:
Is the survey meaningless. Absolutely No. By reading this hopefully the person is wiser (at least he/she will pause and think before making silly rhetoric like "xxx makes the greatest phone in the world" or "yyy makes the worst phone in the world")
How will someone be wiser for reading this survey? By following a generalisation, which offers no real insight into the conclusion, which it presents? Hardly.
As for silly rhetoric, it would seem more silly to suggest that one base their buying decision upon a survey, which offers very little in the way of factual insight into the nature of its research; no comparative analaytical model to draw parallels from; an isolated market sample with little or no consideration for cultural or social factors; a lack of distinction between the end results used to form a conclusion, and which has been sesationalised thus possibly rendering the objectivity of those reading it, with a view to basing a buying decision on it, suspect.
Quote:
OK. So what do you think is the sample size of SE phones within the 5000+ respondents? Is it big enough for you? If you say yes, then I would tell you the same general conclusion from the survey as I've told max (in previous post). If you say no, then see the paragraph below.
Dilemma. Say someone wants to do a comparative survey/analysis of different brand of phones. Lets take a hypothetical figure of 5000 as a good sample size (i think you would agree with me on this number). But since SE market share is only 6%, in order to get minimum of 5000 for SE, you need to have about 80,000 respondents. You tell me which survey can get this kind of respondents short of a population census? So you are saying we can all dismissed all these comparative surveys as being meaningless. imo, you are opening the pandora box whereby anyone can rubbish any survey that awards accolades to a small player like SE.
I'll just rebut this in one I think, rather than leaving them as seperate points.
First of all the small survey sample was in reference to 3, but let's go with the SE point shall we?
We don't know what size the SE sample was in this survey, that's one of the problems!
What if the SE sample was too big? That would then suggest that more faults should be found with SE products because of it's larger contribution to the overall survey. So if it was too big then that skews the results. Unfortunately the survey doesn't inform us of the sample composition.
What if the SE sample was just right? That would seem to indicate that the conclusion is accurate. Well that may be the case assuming that all the other mnaufacturers tested were also at their optimum sample sizes in this survey. Were they? Again, we don't know! It would also assume several other criteria e.g. standardisation of fault reporting; was this undertaken? Was possible bias against or for particular manufacturers taken into consideration? How were faults defined? There are so many questions.
What if the SE sample was too small? If one has a survey sample of 5,000 then one could include 300 SE users, which is 6% of 5,000. This then reflects SE's market share within that 5,000 whilst constituting a sufficiently large number of samples. What you are talking about is conducting a survey of 80,000 people, not 5,000. The target of your survey is SE, you wish to interview 5,000 SE users and you wish to have those 5,000 represent 6% of an overall survey group. Well first of all 6% of 80,000 is 4,800, not 5,000. Secondly the same results can be achieved by using a smaller group. There should be no extreme change in the overall figures. Let me illustrate this with a wee example:
We are interviewing 5,000 people on mobile phone faults. Now if 300 of them use SE handsets and the fault rate of SE handsets is, for example, 20%, then approximately 60 of those SE customers will report a fault. If we are interviewing 80,000 people and 4,800 of them use SE handsets and the fault rate is still 20%, then approximately 960 will report a fault. More faults may be reported, but the ratio is approximately the same.
The underlying fact does not change. If you ask 1 million people a question with a yes or no answer e.g. has your handset had a fault? and the fault rate coming out of the factory is 10%, then approximately 100,000 will answer affirmative. If you ask the same question to 100 people and the fault rate is still the same then the results will remain roughly the same with around 10 people confirming faults. Unless you reduce the sample size to an extreme level e.g. 2 people, the results will be approximately the same assuming that the conditions remain constant.
The basic point about this survey is that whatever the sample size was, we don't know how it was composed. Was there even any thought to reflecting market share in the survey? Was the sample size divided up equally amongst the manufacturers? Furthermore we are also not considering bias on the part of the respondents, were they prejudiced towards certain brands or favourable? The whole survey lacks credibility as anything more than a trend indicator in my opinion.
You are correct, I am rubbishing these surveys, because they are meaningless. They only indicate trends and do not offer enough information to form a factual opinion. Even if everyone in the world was included in this survey it would still be meaningless on a factual level because it would still only indicate a trend. Without providing the fundamental information behind the survey results it is worthless save as an indication of a trend amongst a particular group of people. It tells us nothing of that group of people nor of the details involved.
--
Posted: 2005-09-01 06:51:01
Edit :
Quote
Quote:
On 2005-09-01 05:19:11, Gigs wrote:
One SMALL teency sticking point..
Like I said before, it doesnt matter whether it is network or phone faults.
If you're going to call into question a manufacturers reliability then the above matters a great deal.
Lets say for arguments sake that everyone of the
respondants just happened to be an orange (it was orange wasn't it?) user with a K700i, the txting issue that was at fault within their network would have made an absolute joke of any statistical result.
After all, by that token CD/DVD burners would have horrible reputations.. Given that many are based on the cheapest shoddiest media people can find to use in them. It must however be the units fault, not the media used in it.
well, u have a good point. I'm dont know about the orange problem, but it wont change a thing because phones from other manufacturers in would also experience the same problem and this would even out this anamoly statistically. However, if you say the problem only happens with K700 then the fault lies with SE phones, isnt it.
ditto for your DVD analogy (i.e. other brands dvd player owner would also blame the dvd player).
--
Posted: 2005-09-01 07:11:34
Edit :
Quote
@scotsboyuk
that was a long reply.
Quote:
Well first of all 6% of 80,000 is 4,800, not 5,000.
You are really meticulous arent you?.
.
Quote: What if the SE sample was just right? That would seem to indicate that the conclusion is accurate. Well that may be the case assuming that all the other mnaufacturers tested were also at their optimum sample sizes in this survey. Were they? Again, we don't know!
Do you think the people in Which? who did the survey (and analysed it) is SO STUPID to print something that is statistically undefendable??? I thought there is still libel law in UK. If only someone has the magazine!!.
Quote:Furthermore we are also not considering bias on the part of the respondents, were they prejudiced towards certain brands or favourable?
I thought this is about phone-users
feedback survey and not about QC/engineering survey (of which you need to go to factory production line to get). What if the survey just ask a simple question? "Did you buy a new phone in the previous 12 months. If yes, what brand is it and on which network and did you encounter any fault?".
Any survey of this type would inherently contains the variability that you mentioned. There are thousands of such surveys/polls (from politician approval ratings to tv programme polls) happening everyday. All these may be meaningless to you but some way or others these surveys would have some effect on something.
Quote: The whole survey lacks credibility as anything more than a trend indicator in my opinion.
The survey is saying nothing more than a trend. So I dont know why you want to question credibility. If you agree that the survey gave a trend indicator, then the survey data should be credible enough to derive this trend, nothing more and nothing less.
Quote:Without providing the fundamental information behind the survey results it is worthless save as an indication of a trend amongst a particular group of people. It tells us nothing of that group of people nor of the details involved.
It seems to me you are trying to question the independence of the survey. I would agree with you on this
if and only if you can ascertain that readers of Which? are bias towards certain technology/brand/practice (i.e. something similar to SE-biased esato). Maybe you tell me since you r from UK, you should know better.
--
Posted: 2005-09-01 10:10:12
Edit :
Quote
New Topic
Reply
