Skip Navigation
Shadow Shadow @lemmy.ca

Infrastructure nerd, gamer, and Lemmy.ca maintainer

Posts 98
Comments 1.6K
Which Lemmy web frontend is most popular?
  • This is actually a piece of data I've been wanting to pull too, but it'll probably take me a few days to get to it.

    In the meantime, I'll just let this url out: https://data.fedecan.ca/images/

  • Amazon has already made a Bond film – it cost $300m and is unwatchable
  • Yeah they use the same color grading style or something.

    I've been enjoying all their SciFi: silo, severance, for all mankind, foundation.

  • Amazon has already made a Bond film – it cost $300m and is unwatchable
  • The Boys is decent. Otherwise not really.

    I only really think of Apple as being able to consistently make good shows.

  • We are excited to announce pixelfed.ca, a new Pixelfed instance run by Canadians and hosted in Canada [edited, see comments]
  • Unfortunately I don't think historical posts are pulled in, just things that have happened since our server started watching their server. If you follow them you'll see any future things they make (as will other users on our server).

  • Look at your boxes, Uline is not only American but the wrong kind of Republican
  • I found https://www.whitebird.ca/ on google which seems to be a decent alternative

  • Removing Jeff Bezos From My Bed ◆ Truffle Security Co.
  • I was intrigued by those things a while ago but the subscription + cloud BS was a no-go for me. That aquarium chiller is a cool idea, but not many of those covers on ebay.

  • What a moron
  • They're pretty common in nicer datacenter environments. Newer cat6a is only 28awg (tiny and easy to work with!) and can do 10gb just fine.

    For example:

  • What a moron
  • Nah like this:

    Instead of like this:

  • What a moron
  • Boots suck. Good connectors just have the tab be more of an inverted V shape, so it doesn't catch but you don't need to squish a hard plastic boot to remove it.

  • 70% of Canadians support retaliatory tariffs on United States: poll
  • Cutting off the power we supply to them.

    Isolated places like Point Roberts would be screwed, and the rest of the US would feel it too.

  • Ontario government sponsors Sean Hannity’s show on Fox News for Trump and Musk interview
  • The Ontario government sponsored Fox News host Sean Hannity’s prime-time interview with Donald Trump and Elon Musk, part of the multimillion-dollar advertising campaign the province is rolling out as the U.S. President repeatedly threatens to impose tariffs on Canadian goods.

    While the campaign is expected to cost tens of millions of dollars, the province is refusing to say exactly how much money is being spent on the ads, including the one for Hannity, one of the highest-rated U.S. cable news shows. Fox also did not respond to requests for comment about the cost and frequency of the ads.

    The sponsorship ad aired during Tuesday night’s Hannity, before Mr. Hannity’s pre-recorded interview with Mr. Trump and Mr. Musk, the tech entrepreneur and Tesla CEO who leads the newly created Department of Government Efficiency (DOGE) initiative in the Trump administration.

    “This program is brought to you by Ontario, Canada, your trusted economic partner,” the ad said, accompanied by an “Ontario Canada” logo with a maple leaf. It was followed by an Ontario government commercial that showcases the strong economic ties between the two countries.

    I A spokesperson for Ontario Progressive Conservative Leader Doug Ford’s re-election campaign said the sponsorship is part of the province’s original deal with Fox and that no new money is being spent during the campaign, which will see Ontarians go to the polls on Feb. 27.

    As part of our U.S.-based marketing campaign that’s raising awareness about Ontario’s importance to America’s economic success, Fox offered the province the opportunity to sponsor programming,” Grace Lee said in an e-mail.

    “This was included as part of an overall ad buy that was finalized weeks ago. With President Trump threatening tariffs that would devastate Ontario’s economy, we will do whatever it takes to stand up for Canadians, including by taking our message to U.S. airwaves and the shows we know Republicans and President Trump watch.”

    🤦

  • Need recommendations for Canadian registrar and hosting
  • We just switched lemmy and our fedecan domains over to canspace and I'm impressed so far. Nice ui if a little slow sometime, but support was quick and effective.

  • Moving to /c/pixelfed.ca

    0
    We are excited to announce pixelfed.ca, a new Pixelfed instance run by Canadians and hosted in Canada [edited, see comments]
  • Surprisingly, lemmy won't let me edit his post since we're both admins. He'll update it soon I'm sure.

  • Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Sorry for the brief downtime

    Sorry about the brief downtime there!

    Wasn't related to the new users or high load, but was a combination of two problems:

    1. Our fw02 took over for some reason TBD
    2. Opnsense isn't keeping haproxy configs in sync and fw02 had a broken config

    Still digging into the root cause, but shouldn't be any more issues =)

    5
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Upgraded to 0.19.9

    Just a heads up that we're now running on lemmy 0.19.9.

    Change log here - https://lemmy.ca/post/38913840

    13
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Oops, sorry about the downtime there

    Sorry about the ~20 minutes of downtime there!

    I was working on migrating the lemmy.ca domain out of a personal cloudflare account, into a shared one for Fedecan. Missed one little setting that took me far too long to track down!

    10
    ktla.com Massive protest against Trump deportations forms in downtown L.A. causing ‘major gridlock’

    A massive protest formed in downtown Los Angeles on Sunday in response to President Trump’s crackdown on immigration and emphasis on deportations. The protest, which was organized and promote…

    Massive protest against Trump deportations forms in downtown L.A. causing ‘major gridlock’
    74
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Images were a little broken, but there's now a fix in place.

    Unfortunately the pictrs database migration yesterday seems to have skipped migrating a ton of data.

    Since cloudflare is caching our images, almost nobody noticed. We've now been running on the new instance for over a day, collecting new images into a new incomplete db. Fun.

    I now have two copies of pictrs running, one with the postgres db and one with the legacy sled-db. Requests that fail against our updated one, will be retried against the legacy one. This should result in all images working properly again for now, while I figure out a better long term solution.

    Please let me know if you still see any issues with images loading!

    11

    Former Federal Reserve adviser arrested for passing trade secrets to China, Justice Department says

    More details - https://www.justice.gov/opa/pr/former-senior-adviser-federal-reserve-indicted-charges-economic-espionage

    1
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Defed from lemmit.online?

    I'm curious if anyone here actually finds value in the reddit posts brought over by lemmit.online, since I'd like to defederate from it otherwise.

    It feels actively harmful to lemmy, since so many of the posts it brings over are questions that the original poster will never see. It encourages a conversation that will never happen, so if someone does reply they're going to feel disengaged.

    The bot rarely gets any upvotes or engagement, and I suspect a majority of people (like myself) have just blocked it. TBH I forgot it existed until Tesseract showed me its posts again.

    16
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Tesseract frontend now available

    Hi everyone!

    Tesseract is now available as an alternative front end at https://tess.lemmy.ca/

    22
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Server Maintenance - Jan 29th and Jan 30th at 9AM PT

    Hello everyone!

    I'll be taking the site down for two maintenance windows this week to complete our server migration.

    • Weds Jan 29th - 09:00 - 11:00 PT (12:00 - 14:00 ET)
    • Thurs Jan 30th - 09:00 - 11:00 PT (12:00 - 14:00 ET)

    During the first window I'll be migrating us from OVH to our new dedicated hardware. After this migration there will likely be some temporarily broken images, as it takes approximately 8 hours to resync our object storage from OVH.

    This is a major change and despite my testing, may have some unintended side effects. If you run into any problems that aren't just a broken image, please let us know.

    The second maintenance window is to migrate our pict-rs database from it's local sled-db into our primary postgres DB. This is a much smaller change but since pict-rs checks every image as it goes through them, it takes about 1.5 hours.

    As usual, you can check https://status.lemmy.ca/ for updates.

    35

    Vancouver donut shop Cartems announces closure after 14 years

    7

    Northern lights in Vancouver: Geomagnetic storm watch issued (Saturday night)

    9
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Lemmy.ca update - Finances & New Server

    Hello everyone, we're long overdue for an update on how things have been going!

    Finances

    Since we started accepting donations back in July we've received a total of $1350, as well as $1707 in older donations from smorks. We haven't had any expenses other than OVH (approx $155/mo) since then, leaving us $2152 in the bank.

    We still owe TruckBC $1980 for the period he was covering hosting, and I've contributed $525 as well (mostly non-profit registration related stuff, plus domain renewals). We haven't yet discussed reimbursing either of us, we're both happy to build up a contingency fund for a while.

    New Server

    A few weeks ago, we experienced a ~26-hour outage due to a failed power supply and extremely slow response times from OVH support. This was followed by an unexplained outage the next morning at the same time. To ensure Lemmy’s growth remains sustainable for the long term and to support other federated applications, I’ve donated a new physical server. This will give us a significant boost in resources while keeping the monthly cost increase minimal.

    Our system specs today:

    • Undoubtedly the cheapest hardware OVH could buy
    • Intel Xeon E-2386G (6 cores @ 3.5ghz)
    • 32gb of ram
    • 2x 512gb Samsung nvme in raid 1
    • 1gb network
    • $155/month

    The new system:

    • Dell R7525
    • AMD EPYC 7763 (64 cores @ 2.45ghz)
    • 1tb of ram
    • 3x 120gb sata ssd (hw raid 1 with a hot spare, for proxmox)
    • 4x 6.4tb nvme (zfs mirrored + striped, for data)
    • 1gb network with a 50mbit commit (See 95th percentile billing)
    • Redundant power supplies
    • Next day hardware support until Aug 2027
    • $166/month + tax

    This means instead of renting an entire server and having them be responsible for the hardware, we'll be renting co-location space at a Vancouver datacenter PDF via a 3rd party service provider I know.

    These servers are extremely reliable but if there is a failure, either Otter or myself will be able to get access reasonably quickly. We also have full OOB access via idrac, so it's pretty unlikely we'll ever need to go on site.

    Server Migration

    Phase 1 is currently planned for Jan 29th or 30th and will completely move us out of OVH and onto our own hardware. I'm expecting probably a 2-3 hour outage, followed by an 6-8 hour window where some images may be missing as the object store resyncs. I'll make another follow up post in a week with specifics.

    Phases 2+ I'm not 100% decided on yet and have not planned a timeline around. It would get us into a fully redundant (excluding hardware) setup that's easier to scale and manage down the road, but it does add a little bit of complexity.

    Let me know if you have any questions or comments, or feedback on the architecture!

    42
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    5 minutes of downtime in 30 minutes - Complete

    Morning all!

    I'm going to be taking the site down for about 5 minutes, so that I can get a consistent copy of our databases (postgres + pict-rs sled).

    Will do it at about 10am PST.

    8
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Lemmy.ca downtime Jan 5th/6th - Whew, we're finally back!

    Hey everyone, and happy new year!

    Sorry about that super long downtime there. Yesterday (Sunday) morning at 10:03AM PST our server suffered a physical hardware failure, apparently a power supply failure. Unfortunately despite opening a ticket with our hosting vendor (OVH) a few minutes later and them claiming to have 24/7 support, nobody looked at our ticket until this morning when their phone support lines opened and I called them.

    They've now replaced a defective power supply and we're back online, after ~26 hours of being offline. Some pretty disappointing response times, to put it nicely.

    We're planning to move away from OVH at the end of this month, onto proper enterprise grade hardware that we own and control. This will give us a HUGE boost in server resources and allow us to scale for the foreseeable future, while also giving us the control to resolve problems like this much quicker. Expect another follow up post about this in the next couple weeks once I've put together the migration plan.

    Timeline:

    • Jan 5th 10:03am PST - We get alerts to the server being non-responsive.
    • Jan 5th 10:05am PST - I pull up the console via IPMI and it's completely non-responsive. Attempting to power off / on the server or do anything, does not work.
    • Jan 5th 10:15am PST - Initial support ticket created with OVH. I followed up a couple times over the next few hours, and got no response.
    • Jan 6th 6:32am PST - Called OVH, gave them the case number and asked them to investigate
    • Jan 6th 7:34am PST - I get notified they'll start their "intervention" in 15 minutes.
    • Jan 6th 11:04am PST - Call them again, the tech is still working on it and they'll get back to me with an update
    • Jan 6th 11:34am PST - "I was informed by our data centre technician that there is an issue with the power supply unit for the rack on which your server resides. Your server will come back online once they have replaced the power supply."
    • Jan 6th 12:17pm PST - We're back up finally!

    Edit on Jan 7th @ 8:40am PST: We just had another outage of about an hour. Investigating with OVH.

    59

    Castle Infinity

    0
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    2 hour maintenance window on Tuesday Nov 26th, 8am - 10am PST

    One of the drives in our server has failed. =( Even though it should be a 10 minute job OVH needs a 2 hour window to replace it.

    I've requested they schedule it for Tuesday from 8am - 10AM PST. Hopefully it'll be reasonably quick, but expect cloudflare tunnel errors while they perform the work.

    13
    Lemmy.ca's Main Community @lemmy.ca Shadow @lemmy.ca

    Upgrade to 0.19.7 on Sun Nov 24th at 10am PST - Complete

    Hey All!

    I'm going to upgrade us to 0.19.7 tomorrow (Sunday Nov 24th) around 10am PST. I don't expect significant downtime, but expect a few minutes at least.

    Update: All done!

    15

    The opposite of a void, unless you have treatos

    1

    Schematic problem - It gets hot and smells like burning

    I've been messing around with circuits my entire life but this design was time sensitive and I've never done my own PCB designs before, so I hired someone to put this together. After getting some test boards, when I plug them in the charger chip gets very hot and smells like burning....

    Circuit is just a simple li-ion usb charger and a switch. I've gone through the datasheet for the bq25302 more times than I can count and I'm missing something obvious here. Using it just for delivering power seems to work fine, the problem is only when charging.

    I do see R6 + R7 off TS don't have the recommended 10k values, but I don't feel like that would cause what I'm seeing. This is being connected to a 21700 lipo.

    Someone mind lending me their eyes please?

    bq25302 datasheet - https://www.ti.com/lit/ds/symlink/bq25302.pdf

    !

    8