FORUMS: list search recent posts

Beta Testing

COW Forums : Apple Final Cut Pro X Debates

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Simon Ubsdell
Beta Testing
on Jul 20, 2017 at 9:37:21 pm

How many of you are regular beta testers?

How would you generally rate the experience?

Do you feel the process is more of an imposition than a privilege?

Are you frustrated when your suggestions are not acted upon?

Do you feel as though you are simply being treated as a glorified bug catcher?

Do you have the time in your busy schedule to evaluate the product adequately?

Does your enthusiasm for the process depend on your enthusiasm for the product?

What would make the process more valuable to you?

Do you feel that some companies use beta testing simply to nudge you into talking up their products?

Do some companies make you feel more valued for your feedback than others?

Is any of it a worthwhile use of your time or are you just doing it as a favour?

Simon Ubsdell
tokyo productions
hawaiki


Return to posts index

Dominic Deacon
Re: Beta Testing
on Jul 20, 2017 at 10:34:50 pm

[Simon Ubsdell] "How many of you are regular beta testers?"

It's rare that I'm not beta testing something or other but usually it's for really mundane things that I just I want now rather than waiting for them to released officially. Like controlling Sonos from Spotify. Beta testing actual productivity software is rarer but I do do it, say, yearly.

[Simon Ubsdell] "Do you feel the process is more of an imposition than a privilege?"

Generally the only reason I take on beta testing is because it's the only way to access something that I need/want. So it's definitely never an imposition for me.

At the moment I'm testing Project Felix for Adobe which technically isn't even being called a beta yet. For those who don't know it's an app to texture, light and render 3D models for people who are not 3D modellers. Basically a game changer for me and the most exciting thing I've seen in years. It's interesting to watch Adobes process on this because they appear to be all ears at this point. Really asking the testers what they want conceptually. At the moment, for instance, there are no modelling tools in the program which suits me but people keep asking for them so you can see Adobe really trying to measure the interest levels on certain functions such as this. Personally I think the programs perfect and am rooting for them not to change anything and just make it function better. Others are definitly pushing in other directions though.

[Simon Ubsdell] "
Do some companies make you feel more valued for your feedback than others?"


Well I'm booking a 30 minute questionaire with one of Adobes engineers for which they are paying the princely sum of $50 (in amazon credit). Which seems pretty good to me. In any other beta I'm not even sure if my feedback was read so this is a rare instance. I'm never bothered by the other companies way of doing things though. Like I said I only do it when it's something I want so I figure we're both getting something out of it even if I don't know whether I'm being listened to.


Return to posts index

Mark Smith
Re: Beta Testing
on Jul 20, 2017 at 10:48:40 pm

I beta test for one company that makes products frequently used in the overall workflow of Digital media production. I do catch some bugs and send them alerts, however I know from one of their programmers that I am one of their few beta testers that gives them good feedback on a couple of their products. I have seen more than a few suggestions i have submitted incorporated into their software over the years.
I am not paid , however I always have the latest versions of their software gratis and some of the apps are not cheap. In a few extraordinary cases one of their programmers has bailed me out of an awful jam by whipping out a custom version of an app that solved a particularly sensitive problem I had run into .


Return to posts index


Michael Gissing
Re: Beta Testing
on Jul 20, 2017 at 10:59:57 pm

[Simon Ubsdell] "How many of you are regular beta testers?"

For many years I was both alpha and beta testing software for ADX,Sony, Yamaha, dSP and Fairlight. These were during the early days of the development of DAWs, digital mixing desks and integrating these with video machines. I also was helping with the development and testing of Shotlister, which started as a capture and EDL management tool and became a fully fledged NLE.

All these products I felt needed to be pushed and pulled into certain features plus they also needed early adopters and my involvement was very much to my benefit. Plus it was great fun to be on that bleeding edge. Sure beta testing was sometimes hairy but it meant I developed a reputation for being able to do the impossible and through that I picked up many clients that gave me amazing work for decades.

The alpha testing was with dSP who often wrote software on my DAW which we would compile and test. These people became good friends and we still stay in touch.

I no longer beta test. Indeed I haven't even installed Resolve 14 as I'm mid project on a feature doco. Why? Because after nearly 30 years of it I no longer feel the need or get the buzz that was the heady days of making amazing things work with expensive under powered hardware. To be honest I did twitch when Resolve 14 came out because it has Fairlight under the hood but even with Fairlight, I have shied away from beta testing for many years now.

Yes you have to believe in the product. You have to totally trust the company and the people. Mostly you have to believe you can contribute and that you can develop ideas and feedback that is of use.

These days I think all software is beta. The time it used to take to develop, test and roll out a release version meant most were not exposed to a bug level that is now common.


Return to posts index

Simon Ubsdell
Re: Beta Testing
on Jul 21, 2017 at 12:55:51 pm

Thanks, Michael (and Dominic and Mark) for your thoughts.

From what you're saying and my own experience, I sense that the has been a gradual shift in what beta testing is for and what is expected of the process.

The move towards public betas - and here it's especially interesting to listen to what you say about Resolve 14 - has perhaps eroded the more intimate types of collaboration between developers and testers that used be more common.

Obviously with smaller developers, those intimate relationships do still obtain and obviously even among the larger players there are degrees to which this is true.

I get the sense that public betas are not really betas at all but rather another type of marketing.

Assiduous beta testers such you clearly contributed to the actual shape of the product and I am presuming that that sense of contribution was part of the buzz that kept you doing it. Active beta testing is a lot of hard work but it feels worth it if you get a sense of its value.

Do beta testers these days test less assiduously because they have seen a general diminution in the value of their contribution?

Simon Ubsdell
tokyo productions
hawaiki


Return to posts index

Oliver Peters
Re: Beta Testing
on Jul 21, 2017 at 1:57:32 pm

[Simon Ubsdell] "I get the sense that public betas are not really betas at all but rather another type of marketing."

Probably true. And a massive bug fix experiment. After all, if your work crashes and burns - you were warned. However, when something really egregious pops up, the developers hear about it quickly and that gives them a chance to fix it before the official release.

Of course, the corollary is when software is released for all intents and purposes as a public beta, when in fact it's not labelled as such.

- Oliver

Oliver Peters - oliverpeters.com


Return to posts index


Warren Eig
Re: Beta Testing
on Jul 21, 2017 at 4:59:57 pm

I beta test for a few products. I put them through real world test-- real projects. So far my experience has been the developer really wants to hear my concerns. They are responsive to when I find a bug or make a suggestion to improve the product.

I guess mileage my vary with other's experience.

Warren Eig
O 310-470-0905


email: info@babyboompictures.com
website: http://www.BabyBoomPictures.com



For Camera Accessories - Monitors and Batteries
website: http://www.EigRig.com



Return to posts index

Simon Ubsdell
Re: Beta Testing
on Jul 21, 2017 at 8:05:58 pm

[Warren Eig] "I guess mileage my vary with other's experience."

I have had some great experiences with smaller developers who have a genuine interest in feedback and how it can improve the product.

At the other end of the scale, let's just say it can be a depressing and pointless exercise.

Simon Ubsdell
tokyo productions
hawaiki


Return to posts index

Michael Gissing
Re: Beta Testing
on Jul 24, 2017 at 1:02:44 am

[Simon Ubsdell] "I get the sense that public betas are not really betas at all but rather another type of marketing."

Blackmagic's very public betas are an example of this for sure. However it was Apple that set the most stark example with X. My opinion has always been that FCPX was for many years an exercise in rush releasing alpha software, adding in things to get to a beta stage and then they have done very little further development. Maybe it was just a big loyalty test or an example that there is no such thing as bad publicity as long as the people are talking about the software. This very forum is testament to this new type of marketing.

Adobe is not different. You cannot seriously call software 'release' when reported bugs simply do not get fixed through various versions. The whole point of beta was to catch bugs as well as add useful features. These days it is all about getting feedback from a much bigger cohort about features, so bugs tend to persist as long as they aren't catastrophic. So even though Adobe don't call it beta like Blackmagic, I think Adobe, like everyone is sending out updates that add features at a rate that is simple public beta.

All modern software is in reality a beta form compared to what was normally referred to as release or beta when I was testing for various companies. The advent of public forums like Creative Cow has taken over from private beta feedback.


Return to posts index


Andrew Kimery
Re: Beta Testing
on Jul 24, 2017 at 3:16:39 am

[Michael Gissing] "All modern software is in reality a beta form compared to what was normally referred to as release or beta when I was testing for various companies. The advent of public forums like Creative Cow has taken over from private beta feedback."

I'll add to this the ease of patching via the Internet. I remember around '98 or so the gaming community really started complaining about PC game devs shipping titles too early and then releasing the first round of patches almost immediately. So obviously the bugs were known issues but the publisher didn't want to push the launch date back in order to fix them before Gold Master. And these days it's common for new consoles to require gigs worth of software updates on launch day before you can even pop a game in.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]