[Noisebridge-discuss] PG&E Tools
superq at gmail.com
Sun Mar 4 17:09:11 PST 2012
The login process is simple, but there seems to be some redirection
involved with the actual site that has the data I want.
The basic login URL is 'https://www.pge.com/eum/login' But whenver I
try and post to it, I just get 302'd to the login page. Maybe I need
to fake the UA.
On Sun, Mar 4, 2012 at 12:21, Daniel Pitts <Daniel at coloraura.com> wrote:
> Taking a csv and stuffing it into a data store is an easy problem to solve, which doesn't require a web scrapper at all. If you need help automating the fetching of the csv, chances are you don't need to scrape, just simulate the login process. curl is a useful unix tool for such a thing.
> It is unfortunate that pg&e doesn't have a public restful API for access though. Perhaps a request to them is in order. I've been thinking about that for my own personal purposes.
> Sent from my iPhone
> On Mar 4, 2012, at 10:02 AM, Ben Kochie <superq at gmail.com> wrote:
>> Yes, that's what I'm talking about. We need to grab the files there
>> and import them into a datastore of some kind.
>> On Sun, Mar 4, 2012 at 09:33, miloh <froggytoad at gmail.com> wrote:
>>> On Sat, Mar 3, 2012 at 11:43 AM, Ben Kochie <superq at gmail.com> wrote:
>>>> I've spent some time digging around for open source tools for
>>>> accessing PG&E data. It looks like there is no API or anything useful
>>>> for getting access to the data. We'll have to write a web scraper to
>>>> pull the data. Since this isn't something I'm good at, I'm looking
>>>> for some help. From what I can tell it shouldn't be terribly hard to
>>>> grab the data we need and then feed it into a database.
>>>> Noisebridge-discuss mailing list
>>>> Noisebridge-discuss at lists.noisebridge.net
>>> a web scraper?
>>> for my consumer account I download the usage data in a .csv
>>> you don't see it for the noisebridge account?
>>> something like the following:
>> Noisebridge-discuss mailing list
>> Noisebridge-discuss at lists.noisebridge.net
More information about the Noisebridge-discuss