Hi, This is a high priority ticket and the FFmpeg version is currently used in a highly visible product in Microsoft. We have customers experience issues with Caption during Teams Live Event. Please help
After politely requesting a support contract from Microsoft for long term maintenance, they offered a one-time payment of a few thousand dollars instead.
This is unacceptable.
And further:
The lesson from the xz fiasco is that investments in maintenance and sustainability are unsexy and probably won't get a middle manager their promotion but pay off a thousandfold over many years.
FFMPEG is a core technology. You literally cannot do anything with video without touching FFMPEG at multiple places in the stack.
The fact that we have billions of dollars of revenue flowing through that software every day, but we rely on VOLUNTEERS to maintain it shows exactly how hollow the whole SV entrepreneur culture really is.
Bunch of fucking posers wouldn’t know performance code if it kicked them in the face.
The fact that we have billions of dollars of revenue flowing through that software every day, but we rely on VOLUNTEERS to maintain it shows exactly how hollow the whole SV entrepreneur culture really is.
Exactly: I'm not mad about important things being run by volunteers -- arguably, that's a good thing because it means project decisions are made uncorrupted by profit motive -- but I am mad about the profit being reaped elsewhere on the backs of their free labor.
@grue@vzq this is such an interesting space. The general public has no idea how much of their software relies on open source code and voluntary community contributions. There have been so many attempts to figure out a way to compensate these maintainers, but it doesn't seem like anything has really become the defacto solution. Open Collective and Tidelift are the closest things I can think of.
@grue@vzq The key is that these folks are supposed to have both freedom & power to set direction independent of corporate shit, *and* compensation for their labor.
arguably, that's a good thing because it means project decisions are made uncorrupted by profit motive
Argue-er here, chiming in. This statement could be interpreted as considering only half of the central relationship of capitalism. (Capitalism isn't just about deriving profit from the control of surplus, it's about the relationship between surplus and scarcity. Surplus doesn't mean shit if no one wants what you have.)
The decisions that volunteers make may not be motivated by the desire/ability to make profit, but they can be (and often are) motivated by the opposite; they have to account for the fact that their volunteer work is labor that isn't contributing to their survival -- aka, their day job. The demands placed on them by their other responsibilities will have to take precedence over the volunteer project.
In practice, this means they have to take shortcuts and/or do less than they would like to, because they don't have time to devote to it. It's not exactly the same end product as if it was profit-seeking, since that can tempt maintainers into using dark patterns etc, but they're similar.
Ideally, they would have all the money they needed, didn't have to have regular jobs, but also had families/friends/hobbies that would keep them from over-engineering ffmpeg.
To say this in a simpler/shorter way (TD;DR), their decisions can be motivated by the fact that they aren't making money from it, don't have enough time or resources to do everything they might want.
(Why is this so long?? I'm bored in the train, gotta kill the time somehow..why not say in 1000 words what I could have said in 100)
Those same companies tell you that their products that you paid for don't belong to you. You are just buying a license to use them. Sadly, this asinine concept is spreading even to hardware markets.
I think it's fair to ask them to take their own bitter pill. They should also invest without owning.
the FFmpeg version is currently used in a highly visible product in Microsoft. We have customers experience issues with Caption during Teams Live Event.
This seems like a "you" problem, Microsoft, and since you employ thousands of programmers with the experience to solve your problem and commit the change back to the FOSS project, I think this is also very easily a "you" solution as well.
This is pretty funny, kinda suggests they have no faith in the engineers they work with... ffmpeg is an awesome piece of work, but if it's a bug they can repeat to some level, then like you said, it 100% a them problem!
E: oh, was thinking it was a pm raised it, but seems it was possibly one of their developers, brutal....
On one hand that's fair, but on the other hand Microsoft is the biggest name in software development and ffmpeg is a volunteer gig, this is probably a problem the megacorp can handle.
It's so ridiculous that this isn't even brought up:
The Command you provided worked fine. Thank you so much for the help! Really appreciated!
We are going to proceed to make a release today and test with customers. Will post the updates here.
Gotta love being a forced beta tester... I mean customer.
There are likely other changes made since they released that version to their customers, so the risk is other things in addition to the current thing get broken.
Man, must be rough to be an MS engineer and do work in public. Ignoring the financial aspect, can't say I've never had a similar ticket and resolution.
As if microsoft ever tries to repro anything... Just refer them to some of the most clunkiest and convoluted sites ever, where it should say to just reboot three times and hope for the best.
Hi, This is a high priority ticket and the FFmpeg version is currently used in a highly visible product in Microsoft. We have customers experience issues with Caption during Teams Live Event. Please help,
Use -data_field first as decoder option in CLI. Default value was changed from first to auto in latest FFmpeg version.
Or modify AVOption of same name in API for this decoder.
Thanks @Elon for the reply, This is the command we are currently using:
ffmpeg.exe -f lavfi -i movie=flvdecoder_input223.flv[out+subcc] -y -map 0:1 ./output_p.srt
I will be looking to see any updates in the FFmpeg documentation. Can you please elaborate and provide pointers the right decoding options or the right FF command er can use.
Thank you!
If you've ever been forced to use Teams you must already know they scraped the bottom of their talent barrel for the team that works on it... The software is shit, riddled with bugs to the point where at one point I used to only be able to use teams on my browser because the desktop app just decided to never let me access the text chat, and the browser version I would load it would be a white screen and I would have to refresh 3 times for it to load. But at least it worked after those 3 refreshes. And it was exactly 3 refreshes every single time, never 2, never 4, and 5 was right out. It was always without fail 3 refreshes. Whether loading from Firefox, Chrome, or Edge. Fortunately we don't have too many meetings with people using Teams these days, so I haven't had to use it in a while, but its easily in my top 5 worst software I've been forced to deal with. Maybe Top 3. But its still miles behind Magento. Fuck Magento, just thinking of it right now gets my blood pumping and I refused to work with it ever again about 10 years ago... Fuck Magento. Teams is at least a distant 2nd or 3rd to that. Absolute crap.
I'm convinced it's the whole B-2-B software world at this point. The shit starts at MS (or any of the FAANGS) and rolls downhill to everyone else.
We're working on a huge Dynamics 365 thing at work, and one of the third parties we use for automated testing is just.... the product seems barebones, is clearly built on top of open source automated testing tool, and is riddled with indicators that barely anyone works there, from the AI help bot to the "submit a ticket and we'll assign it eventually" approach to all other interactions.
I looked them up on Linked In and 12 people work there. 8 of them have C-suite or VP titles, and 4 of them are interns from a local university. This is the state of all modern tech: a board room full of investors, a website, and a product barely glued together from FOSS parts by interns. If you wonder why everything feels like a scam now it's because it is.
The first teams was written in AngularJS (which is a slow to run resource hog, but fast to develop) wrapped in Electron. It was kind of a minimum viable product, just to build something quickly to get some feedback and stats on what people needed.
The plan was to build a new native version of teams and build it into the next windows while having an web fallback (built on react) for everyone else.
They stopped working on the original teams and started working on the new versions.
They got half-way through working on the native and react versions when suddenly, covid happened.
They couldn't keep working on the new versions because they wouldn't be ready for a while, so they had to go back and resume development on the old one, introducing patch after patch to quickly get more features in there (like more than 2 webcam streams per call).
Eventually covid subsided and they were able to resume development on the new teams versions.
Windows 11 launched with a native teams version (which has less features but runs super quick), and the new react based teams (which can now be downloaded in a webview2 wrapper) has been in open beta since late last year (if you've seen the "Try the new Teams" toggle, then you've seen this). The React+Webview2 teams will replace the AngularJS+Electron version as the default on July 7th.
So what do microsoft's crack teams working at? typescript? xbox? vscode? Because those are the smoothest microsoft products I tried so far. The rests seem to get the bottom of the barrel these days.
I'm sure Microsoft has some good devs that are a net benefit to the open source projects they use, but this is not one of them.
Found the guy who created the FFMpeg ticket on LinkedIn. Job title: "Principal software engineer at Microsoft", saying they are "A detailed, analytical Software Engineer with Eighteen years of experience". 18 years?! Fuck me dead...
I'm sure they do too, but I've been surprised many times by the former coworkers I've learned have ended up working for Microsoft. To put it politely, they were generally not the best programmers I've ever worked with.
Hilariously the issue was just a setting change in the update, that you can easily change via a command option. They saw thing didn't work, and didn't read the change log at all before asking to pay a one time fee to guarantee it be maintained for them.
The problem is that Microsoft wants to pay that for a permanent "never maintain in a way that breaks caption decoding in any default behaviour we use" with that one time payment.
Its a quick fix on Microsofts end to change a quick flag in ffmpeg. It's also quick on their end to maintain a fork that only changes the default. One time payments for maintenance make open source projects like ffmpeg subject to fail.
Yes, they should have read the update notes. But I don't see much in the way of documentation regarding the data_field cli option in their documentation even now.
"A failure to plan on your part does not constitute an emergency on my part."
Wow now that is a quote I'm going to steal. Wondering if "A failure to understand on your part does not constitute an emergency on my part." has the same punch or is as relevant... anyway, thanks for sharing!
the xz vulnerability was done through a superflous dependency to systemd, xz was only the library that was abused to use systemd's superflous dependency hell. sshd does not use xz, but systemd does depend on it. sshd does not need systemd, but it was attacked through its library dependency.
we should remove any pointless dependencies that can be found on a system to prevent such attacks in future by reducing dependency based attack vectors to a minimum.
also we should increase the overall level of privilege separation where systemd is a good bad example, just look at the init binary and its capability zoo.
The company who hired "the" systemd developer should IMHO start to really fix these issues !
so please hold your "$they have fixed it" back until the the root cause that made the xz dependency level attack possible in the first place has been really fixed =)
Of course pointing it out was good, but now the root cause should be fixed, not just a random symptom that happened to be the first visible atrack that used this attack vector introduced by systemd.
Probably cause the software engineers writing a high level chat app in TypeScript don't have the skills or knowledge to fix a bug in a C++ video decoder.
I think the maintainer just viewed the bug report as tone deaf. Microsoft is a trillion dollar company and apparently relying on this library without a support contract. Then they a open a high priority bug item. The maintainer saying it's unacceptable is them basically saying they won't prioritize any work unless there's an existing support contract and that they don't do one off payments for bug fixes, which I think is fair.
I think the maintainer just viewed the bug report as tone deaf. Microsoft is a trillion dollar company and apparently relying on this library without a support contract.
I think this mentality shows a clear dissonance between how maintainers are licensing their software and what are their expectations in terms of retribution from users of their software.
If they release a software package with a license that explicitly states that they allow the whole world to use it freely without any expectation if return, they cannot complain afterwards that some particular people in the world end up using it.
Likewise for bug reports.
If they want to get paid because the software they have been releasing to be used freely by everyone is being used freely by a specific company then they need to get their shit together and release it under a license where they explicitly state their terms. This is crítical for everyone involved, specially end users, because we need clarity on these terms.
There was no bug to fix, the PM didn't keep up with developments in an (apparently) core dependency and was passing outdated arguments to ffmpeg. The fix was for the project to update how it was passing flags to ffmpeg. They'd rather spend the time opening a ticket on ffmpeg's bugtracker and spend thousands of company money begging ffmpeg to help them, when MS is a massive corporation, is apparently relying on ffmpeg, yet has hitherto established no support relationship and also has developed no internal expertise on ffmpeg
They easily could have opened up the code and looked around to find the problem, or checked the changelog since an update broke it, or just rolled back to the last-known working version until they had time to figure it out, instead they just dumped it on ffmpeg's doorstep like their hair was on fire. FFMPEG's development model is explicitly that they iterate quickly and there are very likely to be poorly documented breaking changes between versions. It's not one you pull a new version of casually.
A trillion dollar company using your product in one of their flagship products without a support contract can fuck right off.
Microsoft should be putting up money via the support contract to support the creators in maintaining and further development of their product.
A one off payment might be technically sufficient, it is not ethically or morally sufficient. And to put it in terms shareholders understand.. support contract is cheaper than the cost of an alternative.
Well it depends on the size of the one time payment. A 6 or 7 figure one time payment would likely get a maintainer to do something. But micro$oft should really be paying a long term support contract for sure.
The maintainer is a human that needs to eat every day, and not just whenever their services are needed. So at least, the sum of money would need to be a few times higher than whatever labour the fix takes.
But then, the maintainer's ability to fix these bugs doesn't come from nowhere. They worked on this project for likely a long time, which would also need to be taken into account when agreeing on a sum.
Further, this would be business to business. And those contracts often include the value that the client gets out of the software. So if Microsoft makes billions from this open source library, then the maintainer's - as a business - should receive a payment that reflects this for the fix.
All that implies that a few thousand is not nearly enough. Maybe 100k and the maintainer would budge.
The maintainer is a human that needs to eat every day, and not just whenever their services are needed.
That's perfectly fine.
But the maintainer is indeed explicitly making his work available to the public for free and without any expectation of retribution of any kind, isn't it?
And this isn't exactly something new or recent or novel, right? That's been going on for many years.
Companies hate giving out cash. Even if it’s for software they critically need.
I think for most cases getting the cash is the easy part, and the hard part is getting all the paperwork in place to validate payments to random external entities. If that was easy, nothing would stop any low-level manager from making cash payments to random users with a GitHub account.
Long term maintenance. Meaning not a simple bug fix but providing support on demand and possibly prioritizing requests by the contract grantor for an extended period.
Fixing a bug for a fee will create a liability and obligations for the developer. Should you mess it up, Microsoft will have no issue burying you to save even just face.
I can see him getting into a long term relationship that could guarantee the projects survival long-term,(and you at least invest some money for a lawyer to tell you what your are signing on for). For something that would get a few months for the project not so much.
Seriously. What part of "BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION." do they not understand?
I think adoption of the JSLint license's ”This software can oly be used for good and not evil" clause would cover that. I hear IBMs lawyers had issue with it lol
Old issue, so why post it now make it sound like MS demands something?
I think it's because of that recent security issue, and then the subject of corporations tithing into open source code efforts instead of just using it for freeish, that grew around the discussion of that security vulnerability.
Tasteless! MSFT can have their armies of skilled people do this instead of leeching off FOSS contributions. It’s just not an acceptable move from a profit-driven entity to expect free labor, regardless of the FOSS philosophy of the project!
To be fair, I'm sure this is a lone developer at Microsoft, not Microsoft as a company. A lot of this still absolutely applies, but it's not Microsoft as a company making an official decision to go ask the FFMEPG guys for free shit.
It'd be nice if the guy had an avenue to go to leadership, tell them about the issue, and just ask them to actually fund the guys to work on it.
Companies like Microsoft should really have a fund for fixing open source projects - it's breeds good will, reduces the cost of development, and they in turn get software for much less cost than if they did it themselves.
Like - we are using project X and I want to request a bug fix, they go - estimate your effort in shirt sizes or points or some shit for you to do it.
A bean counter looks at their scale that directly converts effort to cost they have under the table, and they give you a budget to offer the dev of the software as part of the fix request
That’s fine, but long term maintenance is the main pain for FOSS projects. I am not sure what’s the right protocol here, though. In general, maybe FOSS projects should start a subscription service for big time companies like MSFT. Why not?
I wonder if these trillion dollar companies offer support contracts for astroturfing on social media on their behalf. I can't think of any other way so many people are supporting their sociopathic attitude.
For a lot of people, either they accept "this trillion dollar corporation that controls all my computers, and the programming languages I use, and my code editor, is evil". Or they accept "this trillion dollar company does lots of good things for me and is good".
I am confused. I realize this is just a flag change not even a dev problem but PEBKAC, still - in the event of an actual bug, why wouldn't Microsoft have a dev contribute to the project and fix it instead of just opening a ticket?
The devs don't take an issue with the ticket being filed. They're irritated by one particular reply which sounds like "My million dollar product depends on this bug fix. Please do that for me". MS isn't offering a solution. They're asking for one.
To be fair MS offers an amount for the fix. Most companies just bully the devs instead. However, I don't think it's quite fair (though legal) to offer one time payments for a core library that they use.
Alternative answer:
"We understand your issue and will fix it as time and priorities allow.
Please note that customers paying for support always get higher priority. Given MS contributions to the project, this ticket was ranked 42nd in our priority list.
Hi, This is a high priority ticket and the FFmpeg version is currently used in a highly visible product in Microsoft. We have customers experience issues with Caption during Teams Live Event. Please help,
comment:5 by Elon Musk, 11 months ago
The order of the comment headers is the other way - above the comment it goes with. If you scroll to the top, you can see it better there. The Microsoft person is Zied Aouina
MIT license to make money is bad because of this. You shouldn't make money or ask me for support in first place if you arent sharing earnings bitch. This should be forbidden by law because software is given AS IS.
If you look up Zied Aouina (issue creator), he's a principal SWE at MS. Seems within his power to read the codebase and figure out his question if he claims he can't find the documentation.
after looking at the ticket myself i think the relevant things IMHO are:
a person filed a bug report due to not seeing what changes in the new version caused a different behaviour
that person seemed pushy, first telling the dev where patches should be sent to (is this normal? i guess not, better let the dev decide where patches go or -in this case- if patches are needed at all), then coming up with ceo style wordings (highly visible, customer experience of untested but nevertheless released to live product is bad due to this (implicitly "your") bug)
pushiness is counterparted by "please help"
free-of-charge consulting was given by the one pointing to changes likely beeing visible in changelog (i did not look though) but nevertheless it was pointed out to the parameter which assumes RTFM (if docs were indeed updated) that a default value had changed and its behavior could be adjusted by using that given parameter.
up to there that person -belonging to M$ or not (don't know and don't care) - behaved IMHO rather correctly, submitting a bug report for something that looked like it, beeing a bit pushy, wanting priority, trying to command, but still formally at least "asking" for help.
but at that point the "bug" seemed to have been resolved to me, it looks like the person was either not reading the manual and changelog, or maybe manual or changelog lacks that information, but that was not stated later so i guess that person just did not read neither changelog nor manual.
instead - so it seems to me - that person demanded immediate and free-of-charge consulting of how exactly the switch should be used to work in that specific use case which would imply the dev looks into the example files, maybe try and error for himself just so that that person does not need to neither invest the time to learn use the software the company depends on, nor hire a consultant to do the work.
i think (intentional or not) abusing a bug tracker for demanding free-of-charge enduser consulting by a dev is a bad idea unless one wants(!) to actively waste the precious time of the dev (that high priority ticket for the highly visible already live released product relies on) or has even worse intentions like:
uploading example files with exploits in them, pointing to the exact versions that include the RCE vulnerability that sample file would abuse and the "bug" was just reported cause it fits the version needed for exploitation and pressure was made by naming big companies to maybe make the dev run a vulnerable version on it on his workstation before someone finds out, so that an upstream attack could take place directly on the devs workstation. but thats just creating a fictive worst case scenario.
to me this clearly looks like a "different culture" problem. in companies where all are paid from basically the same employer, abusing an internal bug tracker for quick internal consulting would probably be seen as just normal and best practice because the dev who knows and is actually working on the code is likely to have the solution right at hand without thinking much while the other person, who is in charge of quick fixing an untested but already live to customers released product, does not have sufficient knowledge of how the thing works and neither is given the time to learn or at least read changelogs and manual nor the time to learn the basics of general upstream software culture.
in companies the https://en.m.wikipedia.org/wiki/Peter_principle
could be a problem that imho likely leads to such situations, but this is a guess as i know nobody working there and i am not convinced that that person is in fact working for the named company, instead in that ticket shows up a name that i would assume to be a reason to not rely too much about names in the tickes system always be realnames.
the behaviour that causes the bad postings here in this lemmy thread is to me likely "just" a culture problem and that person would be advised well if told to learn to know the open source culture, netiquette etc and learn to behave differently depending on to who, where and how they communicate with, what to expect and how to interact productively to the benefit of their upstream too, which is the "real price" all so often in open source.
it could be that in the company that rolled out the untested product it is seen to be best practice to immediately grab the dev who knows a software and let him help you with whatever you can't on your own (for whatever reason) whenever you manage to encounter one =]
i assume the pushyness could likely come from their hierarchy. it is not uncommon that so called leaders just create pressure to below because they maybe have no clue of the thing and not want to gain that clue, but that i cannot know, its just a picture in my head. but in a company that seems to put pressure on releasing an untested product to customers i guess i am not too wrong with the direction of that assumption. what the company maybe should learn is that releasing untested and/or unfinished products to live is a bad habit. but i also assume that if they wanted to learn that, they maybe would have started to learn it like roundabout 2 decades ago. again, i do not know for what company that person works -or worked- for, could be just a subcontractor of the named one too. and also could be that the pushyness (telling its for m$, that its live, has impact to customers etc) was really decided by someone up the latter who would have literally no experience at all on how to handle upstream in such situations. hierarchies can be very dysfunctional sometimes and in companies saying "impact to customers" sometimes is likely the same as saying "boss says asap".
what i would suggest their customers (those who were given a beta version as production ready) should learn is that when someone (maybe) continously delivers differently than advertised, that after some few times of experiencing this, the customer would be insane when assuming that that bad behaviour would vanish by pure hope + throwing money into hands where money maybe already didn't help improving their habits for assumingly decades. And when feeding everhungry with money does not resolve the problems, that maybe looking towards those who do have a non-money-dependant grown-up culture could actually provide more really usable products. Evaluation of new solutions (which one would really be best for a specific usecase i.e.) or testing new versions before really rolling them out to live might be costly especially when done throughout, but can provide a lot of really high valueable stability otherwise unreachable by those who only throw money at shareholders of brands and maybe rely on pure hope for all of the rest.
Especially when that brand maybe even officially anounced to remove their testing department ;+) what should a sane and educated customer expect then ?
but again to note, i do not know which companies really are involved and how exactly. from the ticket i do not see which company that person directly works for, nor if the claim that m$ is involved is a fact or just a false claim in hope for quicker help (companies already too desperate to test products before live could be desperate again in need for even more help when their bad habits piled up too long and begin falling on their heads)
I think you've done a fair summary that deconstructed the simple narrative of "evil corporation steals from the poor". Well, for me it did ;)
to me this clearly looks like a "different culture" problem.
That is a key point. To me it is surprising that a developer of such supposed seniority was not aware of (or doesn't care about, or is so pushed for time, or just insensitive to?) the culture differences. That surprise made me jump to conclusions, leading to outrage and frustration.
Deep in my soul I believe Microsoft really is an evil corporation that steals from the poor.
But in this specific instance, your summary made me think of Hanlon's razor.