FORUMS: list search recent posts

Polls

COW Forums : Adobe Creative Cloud Debate

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Tim Wilson
Polls
on Jul 16, 2014 at 10:30:17 am

Threads with my name in them make me nervous, so I'm going to talk about polls here. LOL



My first question is, what do you want a poll to tell Adobe that you think they don't already know?

Is your thinking that, if they really understood how "people" feel, that they'd do business some other way?

I'm not being rhetorical here. What is it that you think can only be said in a poll that isn't being said?

I really care about polls, mind you. I believe in them. But only real ones. You may not believe this, but I actually edit my posts. I just excised 1000 words or so on the details of my background in polling, starting in social work in 1981 and including my time as a political consultant around the turn of the century. (Yes, I am a man of a nonlinear career path.)

That's why I don't take polling lightly. I take it with the seriousness of 20 years experience conducting and 30 years interpreting polls. And that's why I also don't have any respect for online polls.

Specifically, because I know what a real poll is like, I know that we don't have the resources at the COW to do one, and I don't want to participate in an unreliable one.

Which any online poll almost inevitably is.

Most people might want to stop reading there. LOL But if you want to know my personal reasons for feeling this way, please read on.


In the spirit of my boardroom with its empty whiteboard, here are my PowerPoint slide bullet points, followed by the presentation notes:

  • Self-selected internet polls are obstacles to reliable information
  • High level overviews are cheap, easy and instant
  • Results must still be measured against behavior


NOTES:

Self-selected internet polls are obstacles to reliable information

Polls are all about sifting signal from noise. Statistically speaking, people who answer internet polls tend to have an axe to grind. LOL So that's your model of Customer P, the one who answers polls. Axe to grind. LOL

Overstated, but not by much.

But hey, polls are fun, and to a suit who doesn't understand how polls work, you can throw up an internet poll, give 'em the results, and they'll think you've done your job.

A real poll includes science to model the entire universe of potentially relevant people, what it will take to get a relevant sample, and whether the questions themselves, or who's asking them, or where they're being presented, are skewing data. Don't have the science ducks in a row? Then the answers are worthless.

They're actually WORSE than useless, because easy but unreliable results of internet polls lull companies away from doing REAL surveys. Why would I want to do a REAL survey? I already have the results of this one that took me 2 minutes to post! Even if it's not worth MUCH, it's worth SOMETHING, right?

No. You should never rely on unreliable data.

Apply this to your client shooting video and telling you to make a TV show out of it. You'd have been much better off if they'd never shot a frame themselves. Their "work for you" gets in the way of the ACTUAL work you need to do.

Don't ask internet polls to give you useful information any more than you would ask a client to shoot video for you.


High level overviews are cheap, easy and instant

This was one of the real revelations once I started to get into professional social media monitoring. There are products that you can get, and services that you can subscribe to for very short money that will scour twitter and Facebook (among others), and give you close to real time results. You can see RIGHT NOW how many people are talking about you, and what's happening with that information.

So, in the past hour, 24 hours, week or month, this many people mentioned us. This many people mentioned our competitors. These are the keywords associated with us/them. These are the key attitudes.

(These are their computing platforms. These are their media formats. These are the TV shows they watch. More of them have dogs than cats. You tell this stuff to the world. Do you really think Google and the NSA are the only people besides your followers who read it?)

There's actually a word for this kind of view of the world: mindshare. It sounds like a nonsense Liberal Arts word, but it's a real, quantifiable number. You can KNOW how many tweets there are with your keywords in them. You can KNOW which of them are landing with force.

You can then drill down and see WHO said it. The tweets on my topic from people with more than 10,000 followers. Which of those people have tweets that are the most often retweeted/favorited.

Once you know THAT, you can start looking for their other platforms. Imagine a social media maven named, I don't know, let's say Harry Borden. Harry has 10,000 twitter followers, and X% of his tweets are retweeted/favorited at least Y times, so let's track his Wordpress blog.

And then let's watch to see if he understands how to work Google. LOL This is just an example, does not reflect anyone in the real world, but that's the idea.

BTW, I don't KNOW that Adobe is doing this kind of tracking -- but it strikes me as inconceivable that they're not. This is Social Media 101.

Even if you think that The New Guy represents a fundamental break in Adobe's tradition of customer focus, nobody whose eye is on the bottom line is taking their eyes off customers. Intellectual property is your COMPANY's equity, but customers are your BUSINESS's equity. So unless your COMPANY isn't interested in BUSINESS....

My point being that the raw data for mindshare is easy to find. You can get a view of the ENTIRE aggregate in nearly real time, vs the weeks or months it will take to get a proper survey of a proper subset.

Of course, it also begs an even more fundamental question for a business: what's the relationship between the opinions expressed on the internet vs. opinions NOT expressed on the internet? I've been an Adobe customer since 1985, but I only tweet about music. I don't do Facebook. Nobody tracking me online knows anything more specific about me than that I think Pink Floyd guitarist David Gilmour is dreamy. (oooh David, never wash your hair. LOL)

I do in fact let an opinion slip here and there on the COW LOL but I'm otherwise invisible. How many Adobe customers and potential Adobe customers don't talk about Adobe online at ALL?

THAT's where a survey becomes worthwhile. Getting to know people you don't know already. But by definition, it ain't gonna happen on the web. You already know those people, and EXACTLY what they think.


Do note that I think there are good reasons to poll for questions that aren't answerable by constant social media tracking. What's your file transfer method? How are you thinking about the cloud? How big is your storage?

Unless the respondents have been carefully modeled against the whole, though, they can't represent the people who aren't surveyed. They're not an actual sample.


Results must still be measured against behavior

After a proper survey, you've still primarily measured feelings . You haven't done the work to correlate those attitudes with action.

Although hey, maybe that's the only question that you've assigned your poll to answer. How does "The Internet" feel about what we're doing?

For which you don't need a poll. You don't need to wrestle with getting a properly worded, statistically significant sample of online opinions, when you can get THE WHOLE THING.

Actions are also super easy to measure. How many people are signing up? How long are we keeping them?

Which brings me back to the beginning. What information do you want Adobe to have that you think they don't have already?

Is it that you think that if they really, truly understood you, and everyone else who feels the way you do, that they'd do things the way you think they should?


Return to posts index

Aindreas Gallagher
Re: Polls
on Jul 16, 2014 at 12:11:27 pm

[Tim Wilson] "Is it that you think that if they really, truly understood you, and everyone else who feels the way you do, that they'd do things the way you think they should?"

yes. this bit - just this bit - there were other words and sentences before it, But I couldn't make them out.

ho ho.

[Tim Wilson] "You can get a view of the ENTIRE aggregate in nearly real time, vs the weeks or months it will take to get a proper survey of a proper subset... you don't need a poll. You don't need to wrestle with getting a properly worded, statistically significant sample of online opinions, when you can get THE WHOLE THING."

that is a super duper interesting observation that I had in no way internalised as, like, an actual fact.

the only thing that would be cool would be of some kind of wiki body that made those real time analytical social data search trawls publicly available as a non-profit thing. Although that might get a little weird. We're constructing panopticons like its going out of style as it is. although that sense that all the marketers, and sellers and corporations can pore over the internet as we're using it, with preferred access to our social networks and ability to conduct harvest trawls in facebook in ways that the average user never would be allowed is increasingly weird I find. Its like we're all in this nice restaurant and we're forcing ourselves to ignore the ten foot tall men with magnifying glasses standing over our tables and checking our wallets every two hours. And then once every few days the facebook giant experimentally shouts abuse at a single diner until they start crying.

http://vimeo.com/user1590967/videos http://www.ogallchoir.net promo producer/editor.grading/motion graphics


Return to posts index

Franz Bieberkopf
Re: Polls
on Jul 16, 2014 at 12:16:59 pm

[Tim Wilson] "What is it that you think can only be said in a poll that isn't being said?"

Tim,

Thanks for responding.

I read most of it.

I'll respond with one question: what do you think the value of this particular forum is?

Franz.


Return to posts index


Aindreas Gallagher
Re: Polls
on Jul 16, 2014 at 12:33:45 pm

Well, personally I'm in it for self aggrandising posts berating adobe and the ever delicious raw meat of trollish conflict.

http://vimeo.com/user1590967/videos http://www.ogallchoir.net promo producer/editor.grading/motion graphics


Return to posts index

Paul Neumann
Re: Polls
on Jul 16, 2014 at 2:21:51 pm

This forum has the best ads.


Return to posts index

Aindreas Gallagher
Re: Polls
on Jul 16, 2014 at 2:42:07 pm

That too. It's like a dynamic frieze all around the comment area.

http://vimeo.com/user1590967/videos http://www.ogallchoir.net promo producer/editor.grading/motion graphics


Return to posts index


Brian Charles
Re: Polls
on Jul 16, 2014 at 4:06:31 pm

Tim thanks for a terrific thoughtful post, I would like to read the un-edited version. I agree with most of what you wrote.

@ Aindreas – the facebook giant  – so that's who's bellowing at us. We need to circulate his image so more people can spot him.



Return to posts index

Tim Wilson
Re: Polls
on Jul 16, 2014 at 5:48:20 pm
Last Edited By Tim Wilson on Jul 16, 2014 at 6:10:16 pm

Aindreas, your replies to me and Franz may be my two favorite posts in the history of the COW.

Franz, I have several reasons for thinking this forum is very important.

Its first purpose, historically speaking, was to create a venue for this kind of discussion. That may seem small and obvious, but it's actually a significant diversion from the COW's original purpose, which was just-in-time support for pressing problems.

Indeed, the COW was formed NOT by business people, but by video producers in remote locations in the earliest days of the internet. (The COW's predecessor community opened its door just four months after the release of Netscape Navigator, in the days when you were expected to pay for web browsers.) There was no thought that this could be an actual business because, a) there was no such thing as web advertising, and b) they already HAD a business. Video production.

As part of the COW's commitment to preserving the signal to noise ratio of the forums, these discussions had to move out of the main Premiere forum, as the FCPX debate had to move out of the main FCP and later, FCPX Techniques forum.

(Ironically, the "help" forum for FCPX came LATER, because the debate began well before the product was released, yes? People looking for help became the noise!)

Second, the problems being addressed here aren't typically time sensitive, right? They're broader discussions about things like business models, sensitivity to customers, the nature and purpose of archiving, and our great good friends Sen. Paul Sarbanes (D–MD) and Rep. Michael G. Oxley (R–OH-4).

Third, a forum like this skirts many of the problems of internet surveys. There are no poorly designed frameworks that unintentionally serve to keep real information OUT.

On the flipside, it is VERY raw data that needs filtering to be useful in the long run. For example, debate forums attract people who are LOOKING for a debate, and especially in online debates, "no" is always louder than "yes."

But here's a barely filtered result that actually has some use as ONE data point in a constellation of them: the COW's After Effects community is dramatically larger than its Premiere community...but very few people whose primary living is in AE show up in this forum on a regular basis.

Now, it's ridiculous to draw conclusions based on the absence of data, and I don't think for a moment that Adobe would do that...but do consider what one might do with that single data point. You always need a point of orientation, even if it takes more than one data point to actually navigate.

So, apart from any data about the CONTENT of the posts, or who's making them, Adobe can see that heat is centered on Premiere. So can Avid, whose advertising here now includes messaging designed just for the opinions represented here far more articulately than anywhere else on the web.

Finally, forums are the RIGHT PLACE to learn the nuances of the discussion. Twitter is best for sharing links and sending up flares. Facebook is a more advanced blogging platform, minus the nuance, because thumbs up has largely taken the place of comments. You also can't read that ridiculous, tiny blue type. LOL

Here, a discussion can stretch out, and has room to mutate. Mutating discussions lead to a mutating forum. Only a fraction of the posts in the FCPX Debate forum are about FCPX anymore, but it's still valuable as a village green sort of forum. Because that one is there, this forum won't become the same thing, but we do in fact regularly move far beyond the specifics of Adobe and issues specific to their business model.

There's an awful lot to talk about in this business, and few of us have the opportunity to bang around with colleagues or family members who care as much as we do. We find THOSE people HERE.

So, even though this forum has among the fewest people regularly participating (several of you have commented on this of course) and receives among the lowest traffic from outside the COW, it serves a vital purpose WITHIN the COW.

That's just off the top of my head as I leave the office for the day, but I think it's pretty close to the mark. I do apologize that I won't be replying as speedily as usual today, but I'll be back. :-)


Return to posts index

Franz Bieberkopf
Re: Polls
on Jul 17, 2014 at 2:43:46 pm
Last Edited By Franz Bieberkopf on Jul 17, 2014 at 2:56:45 pm

Tim,


Thanks.

While most of what you say focuses on the value of the forum to its users, you want to assess the value of a poll pretty much strictly on its value to communicate to the Adobe board.

I don't understand why that is. (In your imaginary scenario, I do think that a poll would be a good place for a Adobe to start, but that poll and the one Chris suggested for the forum are different things.)

For the record, the poll idea was pretty off the cuff - I didn't realize that CCow doesn't do such things, and it occurred to me to ask.

But I think there is a spectrum of value between the "perfect picture" of users you insist on in one instance, and the individual opinions this forum encourages.

I don't know the value of these forums to Adobe - such questions have come up before. I certainly agree that Adobe CC sales are going to be valued overwhelming more than the opinion of a few participants here (if such things are valued at all). I think the value of these forums is primarily (and maybe solely) to users.

But finally, let's not use sales to indicate any sort of coherent, perfect picture of the user base. It isn't. And it doesn't have to be. It's sales. Making sales targets and understanding the user base are not necessarily the same thing. (And they don't have to be.) Calling it "behaviour" (or behavior) sounds like you've added meaning, and I don't think you have.


Franz.

Edit: and since I haven't done so recently, I'll thank you once again for running this fine establishment.


Return to posts index

Richard Herd
Re: Polls
on Jul 17, 2014 at 4:02:17 pm

Franz, we can still ask those questions and hope to frame the issues:

One question I have is something like this: Since it is well recognized that CC is excellent software, what has prevented your using it:

-- the rental model as not being ownership (this construal includes no perpetual licensing)
-- the rental model as not being capital (this construal is accounting)
-- the price of the rental model


Return to posts index

Franz Bieberkopf
Re: Polls
on Jul 17, 2014 at 4:35:19 pm

Richard,

For myself, it is primarily the first (though I would probably phrase it differently), and to some degree the third.

Franz.


Return to posts index

Chris Pettit
Re: Polls
on Jul 17, 2014 at 4:49:52 pm

[Franz Bieberkopf] "While most of what you say focuses on the value of the forum to its users, you want to assess the value of a poll pretty much strictly on its value to communicate to the Adobe board.

I don't understand why that is. (In your imaginary scenario, I do think that a poll would be a good place for a Adobe to start, but that poll and the one Chris suggested for the forum are different things.)

For the record, the poll idea was pretty off the cuff - I didn't realize that CCow doesn't do such things, and it occurred to me to ask.

But I think there is a spectrum of value between the "perfect picture" of users you insist on in one instance, and the individual opinions this forum encourages.

I don't know the value of these forums to Adobe - such questions have come up before. I certainly agree that Adobe CC sales are going to be valued overwhelming more than the opinion of a few participants here (if such things are valued at all). I think the value of these forums is primarily (and maybe solely) to users.

But finally, let's not use sales to indicate any sort of coherent, perfect picture of the user base. It isn't. And it doesn't have to be. It's sales. Making sales targets and understanding the user base are not necessarily the same thing. (And they don't have to be.) Calling it "behaviour" (or behavior) sounds like you've added meaning, and I don't think you have."


I had busy day yesterday, was looking forward to responding today when I had time.

But Franz just said exactly what I would have said, so I wont repeat his words. ditto


Return to posts index

Andrew Kimery
Re: Polls
on Jul 17, 2014 at 6:07:26 pm

Franz,

Instead of keeping with generalities what if we talk specifics? If there was a poll which forum(s) in the COW would you put it in, what question(s) would you ask and what are the overall goals for the poll/what do you hope to learn from the poll?


-Andrew


Return to posts index

Franz Bieberkopf
Re: Polls
on Jul 17, 2014 at 7:06:11 pm

Andrew,

As I said above, it was pretty off the cuff. I guess it struck me that CCow has a tremendous amount of users passing through and that a good portion of those are going to be Adobe users, and it seems that's an opportunity to garner a general temperature.

It hadn't occurred to me before Chris suggested it and I guess I was surprised at the potential.

As far as "what" - a simple question of preferred software model would be good (rental, purchase, mixed).

But it also seems an opportunity to survey, for example, who uses what NLE software (primary, secondary, colour, sound).

Or other?

Do you have something you think is worthwhile doing here considering the numbers of users and focus and the limitations of what can be accomplished?

It's probably worth noting that I'm pretty sceptical about what surveys really mean - some past examples:

http://forums.creativecow.net/readpost/335/48020
re: SCRI survey

http://forums.creativecow.net/readpost/335/65137
re: Dave Dugdale survey

Franz.


Return to posts index

Andrew Kimery
Re: Polls
on Jul 18, 2014 at 7:10:26 am

[Franz Bieberkopf] "As I said above, it was pretty off the cuff. I guess it struck me that CCow has a tremendous amount of users passing through and that a good portion of those are going to be Adobe users, and it seems that's an opportunity to garner a general temperature."

Whose general temperature would be be taking though? It won't be the general temperature of all Adobe users because we are limiting our scope to just Adobe users on the COW. It won't even be the general temp of Adobe users on the COW because we are further limiting our scope to just Adobe users on the COW that decide to take the poll.

If, for example, 70% of the Adobe users on the COW that decided to take the poll hate CC have we learned anything other than that 70% Adobe users on the COW that decided to take the poll hate CC? It's like when Ann Landers famously asked her readers if they regretted having children. 10,000 responses later Ann was shocked to calculate that 70% said they regretted having children. The following year a poll (a real poll) was taken and 93% said they did not regret having kids.

Voluntary polls are inherently inaccurate because only people that feel strongly enough, one way or the other, will respond and typically very unhappy people are more likely to voice their displeasure than very happy people are likely to voice their joy. I mean, how many people come to a place like the COW and start a thread to say everything thing is working great? ;)


[Franz Bieberkopf] "Do you have something you think is worthwhile doing here considering the numbers of users and focus and the limitations of what can be accomplished?"

From a polling perspective I don't think there is anything useful about the COW (or any Internet forum). From a sharing of information, or just debating for the heck of it, perspective I think the COW is great.


Return to posts index

Tim Wilson
Re: Polls
on Jul 18, 2014 at 7:37:03 am

I'm still curious to hear a single thing that any poll, anywhere, would reveal that isn't already well known.

What is the thing that anybody thinks Adobe hasn't heard in the past year, either in this forum or elsewhere?

Still, it's at least theoretically possible that if there is something unknown that a REAL poll would reveal it....but an internet poll would so thoroughly pollute the data pool that it's better not to do one, and risk confusing the suits who don't understand how polling actually works. Nothing good comes from bad data, and bad data is all that internet polls are capable of, for reasons that I elaborated at length.

So....back to the question. What do you think hasn't come up here or anywhere else on the web, that will only reveal itself for the first time in a poll?


Return to posts index

Andrew Kimery
Re: Polls
on Jul 18, 2014 at 3:45:55 pm

[Tim Wilson] "So....back to the question. What do you think hasn't come up here or anywhere else on the web, that will only reveal itself for the first time in a poll?"

I'd be curious to know how many Adobe users want to burn Adobe to the ground over CC, how many are going to get a 'CC 4 Life' tattoo and how many don't have a strong feeling one way or the other.


Return to posts index

Franz Bieberkopf
Re: Polls
on Jul 18, 2014 at 4:22:06 pm

Tim, Andrew,


The wikipedia references are vaguely interesting.

https://en.wikipedia.org/wiki/Survey_data_collection#Online_surveys

"Online (Internet) surveys are becoming an essential research tool for a variety of research fields, including marketing, social and official statistics research. According to ESOMAR online survey research accounted for 20% of global data-collection expenditure in 2006."

While listing the advantages of online surveys, it also notes:

"... actual survey practice, particularly in marketing research and in public opinion polling, which massively neglects the principles of probability samples, increasingly requires from the statistical profession to specify the conditions where non-probability samples may work."

So the first question (quite off topic, and a garden path all its own) is in what instances non-probability samples work.

But the entry also notes:

"Various robust procedures have been developed for situations where sampling deviate from probability selection, or, when we face non-coverage and non-response problems."

If my understanding is correct, non-coverage and non-response wouldn't be an issue for a CCow survey. The only issue (and the one that both Tim and Andrew focus on) is "probability selection" (ie a representative sample). Both of you state that this is a non-solvable problem, but the wiki entry states there are "robust procedures" to address it. Any thoughts? (I didn't read the linked literature.)

[Tim Wilson] " What do you think hasn't come up here or anywhere else on the web, that will only reveal itself for the first time in a poll?"

Tim, after reading a bit and thinking more about it, I think my first interest now lies in a CCow census.


Franz.


Return to posts index

Andrew Kimery
Re: Polls
on Jul 18, 2014 at 5:31:09 pm

[Franz Bieberkopf] "Tim, after reading a bit and thinking more about it, I think my first interest now lies in a CCow census."

I haven't had a chance to read the Wiki link yet, but I wanted to say that I too wonder be interested in a COW census. While that may also be too labor intensive to be realistic I wonder how difficult it would be to break down how many COW visitors are 'regulars' (posting multiple times a week) and how many are infrequent visitors that really just show up when they are having a problem.


Return to posts index

Walter Soyka
Re: Polls
on Jul 16, 2014 at 8:28:37 pm

If only someone offered a complete set of analytics, social, advertising, targeting and web experience management solutions and a real-time dashboard that brings together everything you need to know about your marketing campaigns...

Oh, wait:
http://www.adobe.com/solutions/digital-marketing.html

(Am I being serious, or tongue-in-cheek? You decide!)

Walter Soyka
Designer & Mad Scientist at Keen Live [link]
Motion Graphics, Widescreen Events, Presentation Design, and Consulting
@keenlive [twitter]   |   RenderBreak [blog]   |   Profile [LinkedIn]


Return to posts index

Rich Rubasch
Re: Polls
on Jul 16, 2014 at 8:42:36 pm

Wait, Tim is leaving the office at 1:48 pm East Coast time? That's trouble. What's he gonna do with the rest of the day? Why do we all have to stay and work until the sun goes down?

Let's take a poll....what time do you check out each day, average?

Rich Rubasch
Tilt Media Inc.
Video Production, Post, Studio Sound Stage
Founder/President/Editor/Designer/Animator
http://www.tiltmedia.com


Return to posts index

Tim Wilson
Re: Polls
on Jul 16, 2014 at 8:53:37 pm

[Rich Rubasch] "et's take a poll....what time do you check out each day, average?"

On average, I NEVER check out. LOL The flipside of being able to work ANY time is that you wind up working ALL the time.

Checking back out now. :-)


Return to posts index

Shawn Miller
Re: Polls
on Jul 16, 2014 at 10:41:30 pm

[Tim Wilson] "[Rich Rubasch] "et's take a poll....what time do you check out each day, average?"

On average, I NEVER check out. LOL The flipside of being able to work ANY time is that you wind up working ALL the time.

Checking back out now. :-)"


Ha ha ha, right. The commute home is just another break in the work day. :-)

Shawn



Return to posts index

Rich Rubasch
Re: Polls
on Jul 17, 2014 at 5:04:19 pm

I do tent to steer the Harley away from the main highway and around thru the farm fields and country roads on the way home. Adds 20 minutes to the commute but adds years to my life!

Rich Rubasch
Tilt Media Inc.
Video Production, Post, Studio Sound Stage
Founder/President/Editor/Designer/Animator
http://www.tiltmedia.com


Return to posts index

Ryan Holmes
Re: Polls
on Jul 18, 2014 at 2:39:45 am

[Shawn Miller] "The commute home is just another break in the work day."

This may become my new motto! Thank you! You win the Internet for today. +1

Ryan Holmes
http://www.ryanholmes.me
@CutColorPost


Return to posts index

Shawn Miller
Re: Polls
on Jul 22, 2014 at 4:54:22 pm

[Ryan Holmes] "[Shawn Miller] "The commute home is just another break in the work day."

This may become my new motto! Thank you! You win the Internet for today. +1"


Thanks Ryan! :-)

Shawn



Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]