Thursday, May 21, 2015

Hadoop - Start Now

About a year ago I began to go down the rabbit hole that they call Hadoop.  We went down this route mainly due to the various vendors we monitor use all different technologies and it seems the only way to get all the required data into one "platform" is via Hadoop.  ETL is the biggest hurdle we would face as each vendor has a different schema and cleaning the data prior to loading it into another database would be cumbersome to say the least (I am on a four person team).

We choose to go with Hortonworks as the Hadoop distro we would use.  I'm not going to lie, the initial decision to go with them was strictly a monetary one.  Everything in the Hortonworks Data Platform (HDP) is open source and free.  Cloudera does provide a lot of features for free, but the ones we would really need cost money and that is something I do not have.  But, also Hortonworks does a very nice job of piecing everything together and their training is top notch.  I have been to their HDP Operations course and my boss along with our DBA have been to their HDP Developer:  Hive and Pig course.

Recently I ran across the following article:  http://www.theregister.co.uk/2015/05/13/hadoop_two_years_of_anaemia/

The long and short of it is, if you are interested in working in big data now is the time to begin.  The barrier to entry is pretty high right now, but jumping in now means you have a two year head start on everyone else when they start making the push to get in.  Job postings continue to rise and certifications are available.  Tie that in with experience in a very new field and you'll be able to write your ticket.

Saturday, January 11, 2014

Geico Digital Insurance Card - Buyer Beware

I recently saw an ad for Geico's Digital Insurance Card.  Basically via the Geico app on your phone you can access you insurance card and present it to law enforcement in the event you are stopped.  Initially it sounds like a great idea.  For quite some time people have been stealing insurance cards from vehicles so this would prevent that.  Also, you don't have to remember to change out your card after renewal.  At face value it seems like a great idea, but in speaking with a friend in law enforcement buyer should beware.

First, you are handing your phone to a law enforcement officer unlocked.  In the event that he/she runs your information they'll have your phone and could go through your phone.  Obviously I'm not saying that they would, but it's just the fact that they could.  Second, in the event of an accident there are two scenarios you'll need to be concerned about.  Scenario A you take your phone with you to the hospital and they are unable to find your insurance card in your car.  You'll be ticketed for lack of insurance and then have to work through the process of getting it fixed.  Scenario B, you present your phone and it becomes evidence in an investigation and thus you cannot have your phone till the investigation is over.

Always stay informed.

Thursday, January 9, 2014

Going to Be More Active

I realize that I need to start blogging a bit more as work has been ramping up and there is plenty to discuss.  Hope everyone is ready!

Friday, November 29, 2013

Bitcoin Reaching Legitimacy (With the Help of Great Vendors)

As many of you know, I am a huge fan of Bitcoins.  While many are in it for the quick buck, I would like to see it gain some legitimacy and the only way for that to happen is to actually spend it.  I have a small mining operation and to date I've generated about $150 USD.  For a long time the biggest issue was how does one spend their Bitcoins?  Any place that took them was either selling illicit goods or wasn't in the mainstream so how do you trust them?  In my research I ran across a company named Gyft, which sells gift certificates and actually accepts Bitcoin.  A few weeks ago I gave it a try and was impressed.

First, I signed up for Coinbase as my online wallet.  My reasoning for this was two fold.  First, I believe they have on of the most secure applications for Bitcoin.  I have it set where a one time passcode has to be entered from my phone in order to get into my account.  Second, it is one of the easiest apps to use for the actual purchasing of items with your Bitcoins.  Once I had everything in place I sent my Bitcoin to my Coinbase address.  I then purchased a gift certificate for Regal Cinemas as I had a plan to see a movie.  Within 15 minutes I had printed off my gift certificate and was a happy man (sadly I did not get to see my move, but that is another story).  With my left over Bitcoin I grabbed a gift certificate for Amazon.  All went off without a hitch!

But as with all good things, sometimes problems arise.  One of the major flaws with Bitcoin is, once the money is sent there isn't a way to get it back.  So I was in the mood for some pizza and got a $25 gift card for Papa John's.  Now I'm not entirely sure what transpire, but it took a long time for Coinbase to confirm my funds.  But they let me send the Bitcoin for the gift certificate, which also took awhile to confirm.  I'm talking over an hour and once it was confirmed I still hadn't gotten my gift certificate.  Now, technically all the Bitcoin I have is via mining so while I was not mad about the lose, it was concerning.  So I contacted Gyft support just to see if there was anything we could do.  Within an hour I got a response.  They showed the payment as still "Paying" and ask that I send them any information I had confirming that the money was sent.

I headed to Coinbase and copied the advanced log of the confirmation of the funds.  The next day I got an email asking I confirm the amount and what I purchased.  With that I was told I would be refunded the money that was lost.  Within 24 hours the Bitcoin was back in my account.  So, go in knowing that you can spend your Bitcoin and at least there is one vendor you can trust.

Sunday, November 3, 2013

Spending Bitcoins

Normally I focus on security topics, but technology in general should always be on this blog.  I've written about Bitcoins before and I am of the mind that they have a bright future.  The price continues to climb and while no one can say for sure what will happen, I figure with the money flowing into via investment it is here to stay.  Litecoin is also beginning to rise and I am behind that as well.  Anyhow, today I wanted to talk about my first purchase with Bitcoin.

I purchased a number of ASIC USB Erupters and a hub to begin to support Bitcoin.  I won't get a return on my investment, but that wasn't the point.  After a month of mining I had finally mined about 0.1 Bitcoins so I decided lets see what I can do with it.  First, I setup a spending wallet with Coinbase.  They allow for easy transfer to USD if that was your aim.  They follow all the money transmitter regulations, so they will perform KYC.  Then you add your bank account and you're ready to go (you don't have to add it if you don't want to).

Another layer of security added is an app to generate a one time code before you can login to your account.  I suggest using it to keep your account nice and secure.  Once that is setup, then you can setup an account with Gyft.  It is a website (and mobile app) that allows you to buy gift certificates with Bitcoins.  They have a decent list of vendors, but it isn't the best just yet.  Amazon is on there and that's about all that I need.

You select your gift certificate, the amount, and then checkout.  You select Bitcoin and it generates a QR or you can choose pay with Bitcoin again.  If using your mobile app, it will pull up the CoinBase app and then the address you'd like to send it too.  Within 15 seconds your funds are sent and within a minute you'll receive your gift card.

Good luck and enjoy!

Wednesday, August 28, 2013

Big Brother....Wrong Big Brother

And I'm back!  Recent job change along with griping about my old job takes a lot of time.

The new job has been interesting to say the least and has amounted to me meeting with a lot of vendors.  Along with doing technical investigations on how their products work, are secured, and the information that they are gathering.  All a stark change from what I was doing and all so very very interesting.  Which leads me to today's post...

In a discussion with a professor not so long ago we agreed that people had named Big Brother incorrectly.  There is a Big Brother, no doubt about it, but I believe we have it incorrectly named.  Most believe it to be the government and while there is no doubt that there is snooping going on they are really more like Big Cousin.  The real Big Brother are the private sector.  The contractors, Facebooks, Googles, cell phone providers, etc.

My talks today lead to finding out the source of the source of the data the vendor was providing.  Ultimately it leads to two or three big companies, but in the past three weeks I've learned how easily they sell that information they gather.  The wealth of information easily accessible for a small price is overwhelming.  All wrapped up in an easy to use web interface because we'd rather not have to trouble you.

Be safe my friends, the Big Family is watching....

Friday, June 7, 2013

P for Privacy

"I know why you did it. I know you were afraid. Who wouldn't be? War. Terror. Disease. There were a myriad of problems which conspired to corrupt your reason and rob you of your common sense. Fear got the best of you and in your panic, you turned to the now High Chancellor Adam Sutler. He promised you order. He promised you peace. And all he demanded in return was your silent, obedient consent." - V - V for Vendetta

I try not to talk politics too often because it's a came you can't win.  Usually both sides are passionate for their cause and will never be swayed to think one way or the other.  But I felt the need to comment on the recent revelation that the NSA received approval to pull meta-data for about 3 million subscribers at Verizon Wireless.  Under normal circumstances this would require that at least one of the parties be outside of the country, but in this case it was allowed to be American to American, all in country phone calls.  Now I will note that since it was meta-data, the only information received was the numbers involved, the length of the call, and location of said call.  They were not recording the substance of the conversation (not that it sets me at ease).

Generally, what would occur, is if enough "evidence" was gathered from the meta-data then an actual warrant for the substance of the conversation would be sought.  I tend to believe that if they believed anything was to the conversations the telco's would merely receive a National Security Letter and probably comply with the request.  Today we learn even more in regards to the fact that it's not just the telco's (don't think AT&T, T-Mobile, et al didn't get the same requests), but several online companies are also providing information of substance.  I think we'll slowly learn how deep the rabbit hole goes at some point.

But my main reasoning for this article is in regards to the fact that I tend to believe something is fowl in the state of Denmark when both parties agree with what is happening.  All week we heard about how everything was above board and legal.  The proper oversight and proper people knew what was happening and had no issues about it.  The proper oversight?  A secret court with judges no one really knows saying yay or nay to what is presented to them.  How exactly do you get your right to a fair trial if you never go to court?  What about innocent until proven guilty?

I laughed because in discussing this with a family member they replied that they had nothing to hide so why worry about?  The issue is with the silent consent you say yes to everything after that.  Slowly more and more rights will be eroded for the sake of security.  One Senator stated that this program stopped one terrorist attack, but didn't name what the attack was.  It's honestly no different then saying the firewall is working without ever checking the logs and being able to show to the boss what was blocked.

It's time we think about exactly how silent we want to be....