Jump to content

SPECIAL: Andrew White of Crowdology Answers your Questions!


Recommended Posts

This week only, Andrew White of Crowdology has volunteered to answer your questions about online survey panels.

Ask Andrew anything you’d like to know about online market research and survey taking, and gain his insights on market research, from the panel’s perspective.

Please post your questions below, and Andrew will answer them. Please keep it respectful and please avoid asking questions about panels besides Crowdology.

Looking forward to an excellent discussion!

P.S. You can find the previous Q & A session here.

Link to comment
Share on other sites

  • Replies 30
  • Created
  • Last Reply

Top Posters In This Topic

  • Crowdology


  • lev


  • 1957horses


  • Administrator


Andrew IIRC has graciously answered questions in the past. Thanks for doing so again. So, now my questions.

1. What is the purpose of these exercises where they have you hit the "e" key for NO and the "i" key for YES and they show you a series of words and you need to rate the displayed words either positively or negatively using the "e" or "i" keys appropriately? And why do they seem to want you to do these exercises until you are totally exhausted and ready to reach into the survey and pull out the survey creator's tongue (if you could).

2. Along the lines of question 1 previously. Does the industry vet any of these survey techniques to see if they are effective? Or do people just make these up to be hip? I find that survey companies like Nielson probably have the most tailored and to the point surveys of all the survey companies. No question seems wasted nor are there any trendy type questions. Just seems like many survey companies shoot from the hip.

3. What are the current trends among survey companies?

4. Why do surveys make you answer endless questions about companies products where you have indicated previously all you have ever done is heard of the product/company? Or never used their product. I know they want your impressions but if I've never dealt with them then all I do is pick the middle answer for every question to try and be neutral. Can't elicit any really valuable data. Some of these surveys ask questions only an insider would have any knowledge of.

5. I guess my theme is the education/background of survey creators in the previous questions. Is there any specific training, certification for survey creators/companies?

Link to comment
Share on other sites

Why is it that sometimes (getting to be reoccurring more now then ever) we complete a survey and the points are not added to our account. Like today I completed one for 35 points went to see them in my account and they weren't there, no points, no you screed out, no it is closed, just page saying ""Thank you for your participation. We value your opinions."??

And times we do complete them but points aren't in my account, so we/I tell them the survey number what it was about, what time It was done and day only to be told we didn't complete it?? When WE/I even send a screenshot proving I/we completed it?? And they still don't credit accounts with the points............... :?

And why are some surveys say 15 minutes to complete and 30 minutes later your still answering questions??

Why when some surveys say you will get an extra (example) 50 points if you use your web cam. Then when surveys are done, we NEVER see the extra points??

Why do some surveys let you get 40-95% done then say "you screened out??

Why do some sites take forever answering questions??

Why do some sites give a 'canned' answer to a question that has nothing to do with the question asked??

What's with the Nelson surveys asking how many time you've gone to see a movie, if you saw a movie you answer yes and you get screened out only to get that same survey again but it is a different number. I've had 100's of those and never complete one................... :shock:

I know I asked this before but I wondered if anything new was found out why they charge exta. but now some sites are having us cash out but they raised the points/dollars in order to cash out. One site we have to make an extra $2. so $22 to get a $20 amazon. Now I know amazons are free and don't charge you. Another site you have to get 1000 points for $50. Now we have to do 1100 points for the same $50. That's an extra 3-4 surveys to complete yet the same $50. That again I get an $50 amazon. Why are we having to pay them to cash out when amazons are free to buy??

I've got more questions. I don't know if you are answering questions pertaining to Crowdology or all/most/some surveys in general..................... :roll:

Link to comment
Share on other sites

Hi Andrew. Thank you for taking the time to answer some of our questions here on this bulletin board. I would like to direct your attention to a thread I started here about 8 months ago. The thread talks about things we wish market research companies would do regarding the collection of survey data and management of panels. Here's a link to the thread:


My question for today is, do you see a trend towards shorter, mobile-friendly surveys as people move towards using mobile devices as their primary internet device? I have seen many companies move towards responsive design in their web presence, yet they are many that still only create surveys that work on or usable on desktops or laptops. If they want to continue to gather data, the usability of the survey design has to be taken into consideration. Gigantic radio-button arrays just don't cut it on an iPhone or Android phone.

Thanks in advance for your answers.

Link to comment
Share on other sites

A lot of people are not able to afford to own a GOOD mobile devices as their primary internet device. You can do surveys on a cheep one but they don't last long, and before you know it you have to pay for a new one. Better to save up and pay for the better made ones. So I hope that 'mobile-friendly' surveys are an addition to desk/laptop surveys.............. :idea:

Link to comment
Share on other sites

A lot of people are not able to afford to own a GOOD mobile devices as their primary internet device. You can do surveys on a cheep one but they don't last long, and before you know it you have to pay for a new one. Better to save up and pay for the better made ones. So I hope that 'mobile-friendly' surveys are an addition to desk/laptop surveys.............. :idea:

Just to clarify - a good survey design should be responsive, meaning it works well on both mobile devices as well as traditional desktops and laptops.

Link to comment
Share on other sites


1) Why the majority of panels (I would maybe exclude just one) don't have procedures in place to ensure that a respondent who starts a survey is allowed to finish it? Rather than being told even at the submitting stage 'quota full'.

The 'First come First served' practise seems also to me the opposite of 'good survey results' as you tend to speed up for fear of wasting time. :cry:

2) When a panel close down or it is sold, panellists earnings are either deleted or sold for less. Any views? :?

3) Relation between Crowdology and 'Cint'. :!:

Crowdology (Redshift Research) but also it says "A panel partner from Cint via PayPal".

Cint seems having several panels, even cookies seem mixing up. Any clarifications please?

Many thanks

Link to comment
Share on other sites

Are surveys slowly being weeded out for online panels ?

Why has the pay decreased on survey sites?

Do you ever see survey companies being a thing of the past? There seems to be a lack in quality surveys and I'm wondering if survey companies are on the decline..............

Link to comment
Share on other sites

There are surveys that seem to be designed for the purpose of promoting a product. They go on and on listing attributes so that by the time you finish, you will be curious is that product is really so wonderful. Is this in fact the reason I sometimes see these? Does a company really contract a survey company for this? Thank you.

Link to comment
Share on other sites


Firstly thank you all for the questions, I will answer them in the order they were received.

JD’s Questions – thank you for the numbers, it makes referencing very easy!

1. I’ll be honest, we’ve not come across this before but it sounds like a psychology style test being applied to market research. It may be to try and tap into your subconscious – perhaps for brand perception or to develop new messages, understand which images are seen in a more positive light etc.

2. The techniques used in surveys are usually heavily tested in academic research and are most likely to have a theory behind them

3. This is a great question, and one we are constantly asking as well. I think there is a trend towards mobile/mobile-optimised surveys to make it easier for respondents to take part. Some companies are moving towards a more interactive and ‘game-ified’ type of survey to try and increase response rates and encourage participation. However, trends are likely to differ between sizes of agency and the sector they primarily work in – for instance, it may not be appropriate to offer a more ‘game-ified’ experience in B2B surveys but is perhaps more welcome in a consumer survey

4. As you say, I believe in this case they are looking for the brand perceptions from those who don’t currently buy the brand/product. The ultimate aim of this may be to identify where they need to change their messaging to improve their perceptions. It may seem tedious if you don’t have experience with the company but if there was a negative reason why you hadn’t dealt with them this would be a good place for them to pick it up (and hopefully rectify it)

5. Most people employed in a survey company will receive on the job training, although may well have a research background and often go on external courses to improve their skills and keep up to date with trends and new survey tools. In the UK the MRS (Market Research Society) provides training and qualifications that include modules for questionnaire design. I would also say that the majority of people who work in market research have a bachelor’s or master’s degree

1957horses – thank you for all the questions, I’ll answer them as best I can

1. Generally what happens is that once the project has been closed and IDs have been confirmed any outstanding points will be added to your account. With Crowdology this can take up to two weeks. I wasn’t aware that it was a growing problem that they weren’t being added immediately, and can only answer based on our own experience, which is that generally incentives are added to the balance immediately, although sometimes technical issues can prevent this process running as it should, and I refer now to the first part of my answer again

2. Similarly to above, if your responses are complete you will receive the incentive when the project is closed and updated (if not immediately) – it is always good practice to keep a note of the information that you mention here (link, time, screenshot) and it would only be in a situation where your responses were not complete that incentives aren’t awarded

3. Surveys times are usually based on the average time taken during a pilot run, or if this is unavailable then some will base the time on the number of questions. In long surveys there are often a lot of conditional questions (that is, questions only asked based on a specific answer to a previous one) – it’s likely that when the survey takes longer you qualify to answer all the questions, whereas on average most would not have to answer every single one.

4. I’m afraid we don’t offer any surveys directly that need a webcam – I suggest you take this up with the support team of the survey company who sent it to you and they can liaise with the survey creators to find out what went wrong

5. This is very bad practice and screening questions should take no more than 3-4 minutes. If you are screened out after four minutes of questions you should take it up with the company who sent it. However, many will aim to keep this to a minimum and all screening questions should be at the beginning of the survey.

6. At Crowdology we aim to answer all support emails within two working days. Some issues take longer to resolve than others as sometimes it is necessary to contact the company that commissioned the survey in the first place and this can take time

7. Sometimes research companies will put in a fake answer to catch out people who are completing the survey without proper thought

8. I’m afraid I can’t comment on this – it may be worth dropping Nielson an email and asking them

9. Some survey sites may be struggling to cope with increased competition within online market research, which may mean they are cutting incentives in order to keep up with demand for lower pricing.

10. If you have questions relating to other survey sites it’s probably best to ask them directly, as we deal only with Crowdology UK and USA so won’t necessarily have the experience to answer them sufficiently. However, if you have any more general questions do let me know and I’ll try and answer as best I can

Jberri12 questions – thank you, we will definitely have a look through that thread and see if there is anything we can do to improve.

In response to your question, I think there is definitely a trend towards mobile-friendly surveys. However, this won’t be instantaneous for all, as software costs for mobile friendly surveys are often much higher so companies will have to make a considered choice if investing in new software. However, I think you are quite right and we all recognise the fact that many would prefer to have more mobile friendly surveys. For example, 14% of visitors to CrowdologyUK are visiting from their mobile.


Thank you for your question. It will often depend on the type of payment you are receiving and who manages the payment system for that particular survey company. For instance, at Crowdology Cint manage our payments for us and we have just introduced a trial of Amazon vouchers as payment, which means our panellists now have the option of either PayPal or Amazon. The Amazon vouchers are being sent out in batches to keep costs down and ensure panellists get to keep all of their incentives. However, this does mean it can take a few weeks for the voucher to be received, depending on which batch you fall into. PayPal is instantaneous, but as I’m sure you know they charge a % of the transaction.

LEO – thanks for your questions LEO

1. This is bad practice – all quota questions should be at the start of a survey so you only have to answer a few questions to find out if your quota is still open. It is upsetting that not all survey companies enforce this – however, it is most likely to be a genuine human error, as most people in this field will know to put these types of questions at the start

2. This is a hard question – if the panel closes down because it is bankrupt/losing money then they may not have the money to pay out. However, from a panellist’s point of view, you have spent the time doing the surveys and I absolutely agree that you should be paid for your time.

3. So, Cint provide us with services such as managing our panellists profile pages and our payment systems in return for access to our panellists. This means that you receive more surveys than you might if we were totally independent and thus earn more. As a small company ourselves it makes sense for us to partner with Cint, as we in turn have access to respondents in other countries or hard to reach areas.


1. I don’t believe this is the case, in fact I would say that online surveys are becoming more popular as more and more people are able to access the internet

2. I can’t answer for sure for other survey sites, but Crowdology has not decreased pay. However, as always there are rising costs and increased competition so I expect some have reduced their payments to panellists in order to become more competitively priced within the market

3. This is an interesting question. Personally I don’t see this happening any time soon, if ever. If you feel you receive a lot of poor quality surveys it may be best to contact those who provide them directly and share your concerns, as no one (panellists and researchers) wants to see a decline in the quality of surveys, either online or otherwise


Thank you for your question. Companies commission market research agencies to find the positives and the negatives so it is unlikely that an agency would construct a questionnaire that could influence and bias results.

Thank you all for your questions, a great variety and it was good to think about issues from a panellist’s point of view. I look forward to more tomorrow!

Link to comment
Share on other sites

Many thanks Andrew for your replies, very interesting.

Q. How independent is actually Crowdology? :?:

The more I tried to find out about panels (eg. wikypedia has quite a bit of history or you find comments here and there on blogs and sum them) the more I get puzzled.

Like cars (I realised this through a survey actually), many names that seems independent brands but at the end there are just a few main corporations behind. The same I started to think about surveys companies. Am I right?

2) How do you describe 'surveys routers'? I do have an idea but more through experience :!:! What kind of companies are these? How would you say panels use them? Connect to them?

- I agree with what you said "all quota questions should be at the start of a survey... to find out if your quota is still open" but :roll: believe me it happens so very often with many companies that quota is reached whilst you may have already answered questions for 20 min... :shock: ....

Thank you.

Link to comment
Share on other sites

This comment is an expansion on a comment I made earlier about some questions almost requiring insider knowledge to answer. At times it seems like with some surveys I am being marketed to rather than the survey gathering my opinions. Any comments on this observation? Not sure how effective it would be as a marketing tool to use surveys given the limited audience but it would seem to be a good target audience given you are screened to take the survey.

Link to comment
Share on other sites

Hi everyone, thanks again for your questions.

Greeneyeddevils – Unfortunately our current Crowdology panels are only open to UK and US residents. This is mainly due to the fact most of the research we conduct is for clients based in these countries. We are always considering starting up new panels for other countries, and Canada is definitely on that list. However, it may be a tricky given the two languages spoken in Canada (our French isn’t that great!!)


1. It is understandably quite confusing, as many panels use each other, some are partnered with Cint so have a similar log in format and invitation style, while others come under a large parent company. Crowdology is owned by Redshift Research, which is an independent business. We use Cint for a few reasons as outlined above, but we do all the recruiting, are first contact for support and make all major decisions.

2. Survey routers essentially allow a respondents who no longer qualifies for the original survey they were invited to be ‘routed’ to another relevant survey. This not only allows panellists to take a survey when they want to, rather than being screened out and having no other options, but also often enable surveys to finish quicker. Please see this link for a fuller explanation: http://www.surveysampling.com/ssi-media ... arch.image

3. It is disheartening to hear that this happens often – please contact the relevant survey support when it does as it is more likely to change when there are many complaints – this should bring the issue home to repeat offenders

JD – This is similar to a question asked above by Cricket – again I find it hard to believe that a company or research agency would use a survey as a marketing tool. It would make much more sense for them to spend the money they are using on the survey to conduct proper market research that in the long term will be more beneficial to their marketing strategy.

Keep the questions coming! :)

Link to comment
Share on other sites

This topic is now closed to further replies.

  • Create New...