Sharing music online rather than by CD, any recommendations?

[OneWorld]

Slightly OT but only a bit...there was a BBC documentary a few days ago about the massive growth of Data Centres. These are the places where your "Cloud" data resides and they use enormous quantities of electricity and water for cooling. There are several centres in Ireland for example and more proposed and it has been estimated that in a decade DCs will account for over 30% of Ireland's electricity consumption.

Every time data goes to the Cloud it stays there until the owner deletes it. Billions of gigs of photos no one will ever see again.

You can put many hours of 24/96 music on a thumbnail sized memory card or give them a USB stick?

Never such a thing as a free lunch.

Dave.

a very good point Dave!

I saw that documentary too, and it was quite thought provoking and in fact confirmed some concerns I had already. The data centres being built in the USA by Meta, Amazon and the usual culprits are the size of Wales!

I just think, with these data centres which will devour energy on an exponential scale, and that together with the increasing number of charger points we will need for EVs It occurs to me we are using more energy in order to go green. When all we need to do is use existing energy supplies in a more responsible way. I remember some time ago doing some work in a council facility where the office windows were wide open and the heating full on!!!!

My partner has always been frugal with energy use, to the point of absurdity, she obsesses about it, but now I am beginning to get her point - and the result? gas and electric bills DOWN by 25% at least!!!!!

And do we really need store all that data currently being shoved up a data centre? I have a personal cloud here at home and it has ballooned to 6TB and I have now committed to and began to just delete. Yes it is hard, the cursor hovers tentatively over the folder, I think shall I create another folder called 'well maybe, or maybe not' it's like taking a pet that's had its day to the vets where there's that heart wrenching moment where you say to the vet "OK, delete it"

[RichardT]

A difference between cloud backup and local backup (depending on what service you use) is the the cloud backups are often kept online at all times for immediate access - whereas local backups can often be kept offline. It's a generalization for sure, but certainly I would not keep the disks spinning if I had to replace GoogleDrive with solely local backups.

For archiving I use Amazon's S3 glacier deep archive service - it doesn't have instant access so I'm hoping that it's held on tape, though Amazon do not reveal anything about how or where they store their data.

[Wonks]

It might be a very good thing Wonks if companies stored their data locally? That would probably be more costly than cloud storage but that means they might manage it better? As it is they can throw billions of GBs into the centres and forget about it but the centre's systems still have to retain it.

Except if you want data security, you need that data stored in at least three different physical locations. So single-office owning companies don't have any option but to pass off their storage to 3rd parties.

And modern management would much prefer to have other companies run their data storage than employ people themselves. And in big city centres, the cost per square meter of office space makes it unfeasible to have that space occupied by computer storage that can easily be sited out of town at 1/10th of the cost.

As data centres can be anywhere, allowing them to be built in very hot areas where cooling towers are the only viable way to cool the chillers is pure madness IMO. But (in the US) that's down to the local state planning laws which we here can do nothing about. If they want to kill their own water supply system, we can't stop them.

[OneWorld]

However, data centres don’t just provide cloud storage. And if that data wasn’t stored in a data centre where the equipment can be very efficiently cooled, it would have to be stored in data centres in office buildings where cooling would be less efficient. The electricity consumption to operate the servers would be the same.

Given that we need data storage, there are advantages to well designed and implemented data centres.

But how many server farms are there running solely to provide for crypto currencies? If you want to start throwing stones, I’d start there.

....................

I was in a solicitors office a few months ago and they had a large paper file cabinet and the drawers were labelled "destroy in Jan 2025" etc. Obviously there is some legal reason why they have to keep physical files for a certain period but realize that they would be snowed under if they kept them forever!
..........................

Dave.

Solicitors along with other professionals - architects, medical etc have to keep records for at least 6 years in case there are legal cases brought up at some point after the details are first filed. I once worked at a civil engineering company and a whole room about 10ft x 10ft was stuffed full of files. There was a similar situation at a Financial Advisors I did some work for though they went to the expense of having the lot scanned and put onto CD. And now I guess they'll have to employ yet another contractors to put all the stuff on the cloud!

[Wonks]

6 years? At least 20 years for architects and engineers. And with legal professions, I suspect it depends on the type of case or legal work involved as to how long. I’d imagine criminal case documentation would have to be kept well beyond any conviction release date in the event of an appeal or new evidence coming to light and the old documentation has to be re-examined.

[OneWorld]

6 years? At least 20 years for architects and engineers. And with legal professions, I suspect it depends on the type of case or legal work involved as to how long. I’d imagine criminal case documentation would have to be kept well beyond any conviction release date in the event of an appeal or new evidence coming to light and the old documentation has to be re-examined.

You missed your vocation in life, what you doing here, you should have been a lawyer

[jimjazzdad]

Except if you want data security, you need that data stored in at least three different physical locations. So single-office owning companies don't have any option but to pass off their storage to 3rd parties.

And modern management would much prefer to have other companies run their data storage than employ people themselves. And in big city centres, the cost per square meter of office space makes it unfeasible to have that space occupied by computer storage that can easily be sited out of town at 1/10th of the cost.

As data centres can be anywhere, allowing them to be built in very hot areas where cooling towers are the only viable way to cool the chillers is pure madness IMO. But (in the US) that's down to the local state planning laws which we here can do nothing about. If they want to kill their own water supply system, we can't stop them.

In the 70s & 80s it was all mainframes and storage was on magnetic tape. Big corporations had data storage protocols essentially the same as big studios. Then came personal computers, micro-computing, the internet and now server farms and bit-coin mining. Big corporations and government still back up to magnetic storage. Everything old is new again. For most of us, cloud storage is about convenience, not necessity. Storage in three different physical locations is for triple jeopardy threats. Only fly-by-wire aircraft, spacecraft, and extremely high-risk activities where life is at stake need triple redundancy. Backup in two locations is plenty for the majority of data producers. HDD and SSD are cheap these days. There is no reason the average studio can't back up to stand-alone storage that can be moved to another secure location. Magnetic and solid state storage drives can sit powered down for indefinite periods. A 2TB HDD sitting on a friend's bookshelf is probably all the backup most of us realistically need. In any case, its all I need.

[RichardT]

For me, the benefit of Google Drive is instantaneous offsite backup - that’s a feature that’s hard to achieve by moving backup media off site.

[jimjazzdad]

For me, the benefit of Google Drive is instantaneous offsite backup - that’s a feature that’s hard to achieve by moving backup media off site.

Yes, convenient and instantly gratifying. But do you really know where your data resides or what the carbon footprint is?

[jimjazzdad]

Back to the OP's question. For sharing or monetizing music, you really do need to upload to a 'cloud' somewhere. Bandcamp or Soundcloud would seem to make the most sense as they are optimized for music storage and sharing. Presumably they have optimized their infrastructure too, as successful businesses do...

[ef37a]

Now of course, being a V old valve amp jockey I know next to Jack S about all this stuff but others and betters have thrown up some interesting points.

Law firms and so on MIGHT need to keep documents 6,10, 20 years but they surely do not need access to them in milliseconds? As has been said, a hard drive uses no power until you need it but the data farms have to keep it 'live' unless you tell them differently. That, it seems to me is the killer, the fact that the centres have to keep the data available for instant access.

Yes, I was surprised that they build the centres in hot places but then the poles are heating up even faster than the rest of the world so maybe the lesser evil? But of course, all that heat eventually ends up in the sea.

The politicians will never tell us but the only way to curb global warming is to use a lot less of everything. (dark side of the moon? But then they would never go for the 2.7sec'ish latency!).

I have a very old NAS drive and the e'tekky in me did not like the temperature it ran at. I found it could be made to spin down and be awoken when needed. Takes about ten seconds...Works for me.

Dave.

[OneWorld]

Now of course, being a V old valve amp jockey I know next to Jack S about all this stuff but others and betters have thrown up some interesting points.

Law firms and so on MIGHT need to keep documents 6,10, 20 years but they surely do not need access to them in milliseconds? As has been said, a hard drive uses no power until you need it but the data farms have to keep it 'live' unless you tell them differently. That, it seems to me is the killer, the fact that the centres have to keep the data available for instant access.

Yes, I was surprised that they build the centres in hot places but then the poles are heating up even faster than the rest of the world so maybe the lesser evil? But of course, all that heat eventually ends up in the sea.

The politicians will never tell us but the only way to curb global warming is to use a lot less of everything. (dark side of the moon? But then they would never go for the 2.7sec'ish latency!).

I have a very old NAS drive and the e'tekky in me did not like the temperature it ran at. I found it could be made to spin down and be awoken when needed. Takes about ten seconds...Works for me.

Dave.

Oh for the days of yore when on Saturday morning I would spin down the road, go to the record shop, spend ages picking out an album, then get my pocket money out, hand it over and walk back up the road as briskly as could be with a new album under my arm, scurry off home and get it on the record deck, I was in raptures, sometimes bopping along, sometimes sat in contemplation, whatever the music inspired. Downloading, yep, it’s convenient, but somehow buying music has lost its sense of adventure and occasion

[Mike Stranks]

Law firms and so on MIGHT need to keep documents 6,10, 20 years but they surely do not need access to them in milliseconds?

Dave.

Indeed!

In the latter days of paper, the hospital for which I worked was storing all of its 'must legally keep for a long time' notes in a secure bunker-like complex a few miles up the road. We could retrieve anything, but it was a few days' turnaround.

I don't know how the NHS is progressing with digitising - it was just beginning when I left - but it is a mammoth task. I was involved in an early pilot in one specialty in one hospital, but the amount of work involved was gigantic.

I suspect they, and other large paper-generators, have taken a pragmatic approach and gone digital from a specific date; scanned perhaps 3-5 years of recent archive, and kept the rest as paper in secure storage, knowing that as the clock ticks, the amount to be stored will be less and less.

As for digital security in NHS systems - the amount of security, resilience and redundancy was impressive enough in my day. One hopes those standards have been maintained...

Oh! I use Backblaze, so don't need to fret about backups...

[ef37a]

Hi Mike, from what I see it ain't good. I was in Integrated Surgery a few months ago and my records amounted to a stack of A4 papers six inches high.
I often saw nurses and admin folk moving files around in supermarket trolleys!

I seem to recall one of the obstacles to digitizing folk's files is getting individual permission? I say advertise it well, do it and let those with a problem opt out?

Then again maybe the fact that touch screens are a super way to spread bugs has only fairly recently been realized?

Dave.

[Drew Stephenson]

A few thoughts.

On data retention: GDPR sets out quite clear guidelines about what personal data can be stored and for how long (and that it must be deleted once that period is over). But some of those data relationships can last a long time; take your pension company for example. You could be paying in for 30-40 years, and it could be paying out for another 20 or 30. Add the requisite 6 years on the top afterwards and you've got a lot of built up data.

On data centres and the environment: here's an interesting thought to add to the mix. If our current number of cars (globally) were all converted to self-driving vehicles, we'd need the same amount of data centres that we currently have (globally) just to process that automotive data. The future of transport has to be active travel (walking / cycling), public transport, then EVs - but almost every government is going about this in the opposite order.

On NHS data: centralising and digitising medical records has been in and out of the plan since at least Blair's time. Unfortunately successive governments have given successive contracts to disparate groups of private bidders with the usual messed up incentives and subsequent political meddling. So it's a complete mess. I think the most egregious of the lot was the recent attempt by Hancock to bundle it all up with an American organisation who were subsequently going to make all the info available for sale. "Anonymised" allegedly but that's a bit of a red herring.

[ef37a]

A few thoughts.

On data retention: GDPR sets out quite clear guidelines about what personal data can be stored and for how long (and that it must be deleted once that period is over). But some of those data relationships can last a long time; take your pension company for example. You could be paying in for 30-40 years, and it could be paying out for another 20 or 30. Add the requisite 6 years on the top afterwards and you've got a lot of built up data.

On data centres and the environment: here's an interesting thought to add to the mix. If our current number of cars (globally) were all converted to self-driving vehicles, we'd need the same amount of data centres that we currently have (globally) just to process that automotive data. The future of transport has to be active travel (walking / cycling), public transport, then EVs - but almost every government is going about this in the opposite order.

On NHS data: centralising and digitising medical records has been in and out of the plan since at least Blair's time. Unfortunately successive governments have given successive contracts to disparate groups of private bidders with the usual messed up incentives and subsequent political meddling. So it's a complete mess. I think the most egregious of the lot was the recent attempt by Hancock to bundle it all up with an American organisation who were subsequently going to make all the info available for sale. "Anonymised" allegedly but that's a bit of a red herring.

As I say Drew, this is all beyond me so I have to come at it like "the man on the Clapham omnibus". Pension: why, how should that generate a zillion bytes of data? I have the state pension, a fixed amount paid into my bank every month, never changes so cannot take much computing power to do? Then I have a much smaller private pension. Again, fixed amount every month. Once a year I get a resume from both which comes to 2 or 3 pages of A4 and fewer than 1000 words. A kByte? Of course there are millions of "mes" but I suspect the whole lot of data could be stored on a 1TB hard drive?
Then there is the time factor again. I cannot see why anyone would need less than one minute to grab someone's pension data?

Self drive cars: Yes I can see they would need vast amounts of real time processing but from what I read, true self driving personal vehicles are decades away. Along with electric cars, "they" have developed this arse 'uppards. We need reliable clean cheap electric buses which, being on a fixed route should be easier to automate?

Dave.

[Mike Stranks]

On NHS data: centralising and digitising medical records has been in and out of the plan since at least Blair's time. Unfortunately successive governments have given successive contracts to disparate groups of private bidders with the usual messed up incentives and subsequent political meddling. So it's a complete mess. I think the most egregious of the lot was the recent attempt by Hancock to bundle it all up with an American organisation who were subsequently going to make all the info available for sale. "Anonymised" allegedly but that's a bit of a red herring.

Sorry Drew, but homing-in on the 'commercial contracts' issue doesn't do justice to the big picture.

A few nuggets of what happened, from my own experience:

Patient Admin IT Systems have been common in NHS hospitals since the mid 1970s, with GP systems of various flavours common from about the same time.

The problem which the Blair initiative sought to address was that the clinical side was largely still paper-based and that the data held by Practices and hospitals was 'locked-in' and largely, technically, unshareable. Changes to both these issues were, to varying degrees, resisted by many clinicians. To my personal knowledge, several hospitals had successfully implemented such all-discipline systems - purchased from commercial suppliers - long before the Blair initiative started in the early 90s.

The Summary Care Record was one aspect of the Blair NHS 'National Programme for IT'. It's relatively modest - and sensible - objectives to hold a very small subset of patient data at an 'England' level were fiercely resisted by many clinicians - both in GP and hospital-land - and by such august publications as the Daily Telegraph and the Daily Mail. (N Ireland, Scotland and Wales have their own NHS systems and are 'independent' in that sense.) In 2007/8 I spent a considerable amount of time fire-fighting with patient-groups and some clinicians, the nonsense being spouted in some sections of the press. The general conclusion, when presented with facts rather than hysteria, was that the proposals were reasonable and sensible. However, many people and clinicians did not avail themselves of the opportunities to be 'deMailed' and thus a fierce tussle ensued about 'Opting Out' or 'Opting In'.

It wasn't (and probably isn't) all the Government's fault - although some of it was and is!

[Drew Stephenson]

As I say Drew, this is all beyond me so I have to come at it like "the man on the Clapham omnibus". Pension: why, how should that generate a zillion bytes of data? I have the state pension, a fixed amount paid into my bank every month, never changes so cannot take much computing power to do? Then I have a much smaller private pension. Again, fixed amount every month. Once a year I get a resume from both which comes to 2 or 3 pages of A4 and fewer than 1000 words. A kByte? Of course there are millions of "mes" but I suspect the whole lot of data could be stored on a 1TB hard drive?
Then there is the time factor again. I cannot see why anyone would need less than one minute to grab someone's pension data?

Well, firstly I was just talking about data retention durations here rather than amounts, just to point out that some places do need to hold that data for a very long time.
But on the actual data, let's take private pensions: it's not just your contribution that needs to be held, it's every investment that's related to it. Your contribution goes into a fund (or funds) which subsequently invest in a whole range of other things. Every transaction (date, time, amount, buy and sell price, transaction fee, commission etc) needs to be recorded and the proportional amount registered against your account. That data does scale up quite quickly. Then you have the usual added fun and games of mergers and acquisitions of both the investments and investors that inevitably lead to duplications and partial duplications across partly-compatible systems.
But I was really just talking duration.

Self drive cars: Yes I can see they would need vast amounts of real time processing but from what I read, true self driving personal vehicles are decades away. Along with electric cars, "they" have developed this arse 'uppards. We need reliable clean cheap electric buses which, being on a fixed route should be easier to automate?

Absolutely, but we in the west have an incredibly car-centric culture. From the weird behavioural aspects that mean people who will happily (well, maybe not happily, but regularly) jam themselves onto a train or tram but wouldn't be seen dead on a bus, to our ability to excuse all manner of toxic or illegal behaviour when it involves a motor vehicle that we wouldn't tolerate elsewhere.
We need to walk and cycle more, when we can't do that we need to take public transport more, and when that's not available, then we need to consider personal EVs. But we have 70-odd years of cultural inertia to overturn. Even staring down the barrel of a climate crisis that will take decades to shift.
And it won't happen at all as long as we continue to over-invest in cars and under-invest in the alternatives.

I blame the press!

[OneWorld]

A few thoughts.

………. I think the most egregious of the lot was the recent attempt by Hancock to bundle it all up with an American organisation who were subsequently going to make all the info available for sale. "Anonymised" allegedly but that's a bit of a red herring.

Ok, it’s a “I have a friend who has a friend……” case, but my source is a trusted source with impeccable credentials. His colleague worked for the organisation and he gave the impression the whole thin* was a load of balloons, with lucrative contracts being dished out like smarties. What was also much acknowledged is privatisation of the nhs is being done piecemeal and it’s being bought up by the Americans. No leader of a government is going to stand at the Despatch Box and announce “We are going to privatize the NHS” so it is being done insidiously. Ever tried to get an NHS dentist lately?

I had some treatment done in hospital, a few days later I was called by a company working with the NHS asking questions about my ‘Journey’ I thought he meant which bus did I take to the hospital. But no, he meant how did my treatment at the hospital go. I said I was happy to answer the questions as long as he assured me my data would not go to a third party or no one but medical staff would have sight of my details.

A few days later my email inbox was festooned with all manner of snake oil claptrap and miracle cures, even medical insurance, all from the US.

[OneWorld]

Seems like Big Data is with us as a matter of fact. Anyone that can come up a cpu that doesn’t run hot enough to fry eggs will have a fortune that even Musk could only fantasise about.

[ef37a]

Seems like Big Data is with us as a matter of fact. Anyone that can come up a cpu that doesn’t run hot enough to fry eggs will have a fortune that even Musk could only fantasise about.

Big data? Yes, I see that but...Pensions again. Not that long ago (to me!) it was all done on paper and in any case we are only talking names and numbers and transactions not billions of bytes of data from millions of distant galaxies or one second of the data from the Cern Collider.

I suspect the people that run and earn from server farms are perpetuating their own existence?

Anyone with half a brain knows the problems but every politician just keeps saying "growth,growth, growth".

Dave.

[OneWorld]

Seems like Big Data is with us as a matter of fact. Anyone that can come up a cpu that doesn’t run hot enough to fry eggs will have a fortune that even Musk could only fantasise about.

Big data? Yes, I see that but...Pensions again. Not that long ago (to me!) it was all done on paper and in any case we are only talking names and numbers and transactions not billions of bytes of data from millions of distant galaxies or one second of the data from the Cern Collider.

I suspect the people that run and earn from server farms are perpetuating their own existence?

Anyone with half a brain knows the problems but every politician just keeps saying "growth,growth, growth".

Dave.

I suppose we are acquisitive by nature and even data, or more to the point data that is the electronic equivalent of flotsam and jetsam has some value. I am currently going through the process of trying to jettison old tunes or more precisely attempts at tunes, and wipe them off the drives, but it seems such a wrench, these days, it’s about the bravest thing I do, press the delete button lol

But I have to admit to myself, a good 90% of the stuff on my NAS drives is bobbins. Politicians trot out the word ‘growth’ as if it’s some mantra forgetting that weeds grow as well as flowers

[Drew Stephenson]

Sorry Mike, I missed your post earlier. I defer to your hands on experience, I was only reading about it from the outside.

[ef37a]

Seems like Big Data is with us as a matter of fact. Anyone that can come up a cpu that doesn’t run hot enough to fry eggs will have a fortune that even Musk could only fantasise about.

Big data? Yes, I see that but...Pensions again. Not that long ago (to me!) it was all done on paper and in any case we are only talking names and numbers and transactions not billions of bytes of data from millions of distant galaxies or one second of the data from the Cern Collider.

I suspect the people that run and earn from server farms are perpetuating their own existence?

Anyone with half a brain knows the problems but every politician just keeps saying "growth,growth, growth".

Dave.

I suppose we are acquisitive by nature and even data, or more to the point data that is the electronic equivalent of flotsam and jetsam has some value. I am currently going through the process of trying to jettison old tunes or more precisely attempts at tunes, and wipe them off the drives, but it seems such a wrench, these days, it’s about the bravest thing I do, press the delete button lol

But I have to admit to myself, a good 90% of the stuff on my NAS drives is bobbins. Politicians trot out the word ‘growth’ as if it’s some mantra forgetting that weeds grow as well as flowers

I seem to have been struck by the "vanishing thread gremlin" yesterday? I said One that for about 40 quid you could get a 2TB USB drive and thus never delete anything? Mine is about 30cu cms so you could store a lifetime's digital dross in a shoebox! And zero hit to the water or lektrik supply.

Dave.

[OneWorld]

Seems like Big Data is with us as a matter of fact. Anyone that can come up a cpu that doesn’t run hot enough to fry eggs will have a fortune that even Musk could only fantasise about.

Big data? Yes, I see that but...Pensions again. Not that long ago (to me!) it was all done on paper and in any case we are only talking names and numbers and transactions not billions of bytes of data from millions of distant galaxies or one second of the data from the Cern Collider.

I suspect the people that run and earn from server farms are perpetuating their own existence?

Anyone with half a brain knows the problems but every politician just keeps saying "growth,growth, growth".

Dave.

I suppose we are acquisitive by nature and even data, or more to the point data that is the electronic equivalent of flotsam and jetsam has some value. I am currently going through the process of trying to jettison old tunes or more precisely attempts at tunes, and wipe them off the drives, but it seems such a wrench, these days, it’s about the bravest thing I do, press the delete button lol

But I have to admit to myself, a good 90% of the stuff on my NAS drives is bobbins. Politicians trot out the word ‘growth’ as if it’s some mantra forgetting that weeds grow as well as flowers

I seem to have been struck by the "vanishing thread gremlin" yesterday? I said One that for about 40 quid you could get a 2TB USB drive and thus never delete anything? Mine is about 30cu cms so you could store a lifetime's digital dross in a shoebox! And zero hit to the water or lektrik supply.

Dave.

"Vanishing thread gremlin" Maybe SOS's cloud is ready to burst, or the Americans have blown it out the sky

Yep I take your point about how cheap space is. I managed to snag myself a Western Digital NAS box with 4TB mirrored drives +5TB standalone which can be connected to the NAS by USB. The NAS box can also be accessed over the internet, my micro-cloud.

So I have concluded that before I press them into active service I am going to do a spring clean of my existing drives and the drives I just mentioned should be all I'll ever need. Well that's the plan anyway. Might be better just to create a new file/folder structure so I can drop on any file in an instant. At the moment my right hand doesn't know where my left hand is, and vice-versa