How can I get content from website using httparty?

Question

I want to grab content from a website, that I input into a submit form, and store that info as a json I can save to my db. I am trying to use HTTParty, but I'm not quite sure how to implement it to grab the data. Here is what I have so far.

controller

  class UrlsController < ApplicationController
  before_action :set_url, only: [:show, :edit, :update, :destroy]
  #require "addressable/uri"
  #Addressable::URI.parse(url)

  # GET /urls
  # GET /urls.json
  def index
    @urls = Url.all
  end

  # GET /urls/1
  # GET /urls/1.json
  def show
  end

  # GET /urls/new
  def new
    @url = Url.new
  end

  # GET /urls/1/edit
  def edit
  end

  def uri?(string)
    uri = URI.parse(string)
    %w( http https ).include?(uri.scheme)
    rescue URI::BadURIError
      false
    rescue URI::InvalidURIError
      false
  end

  # POST /urls
  # POST /urls.json
  def create
    @url = Url.new(url_params)
    @app_url = params[:url]

    respond_to do |format|
      if @url.save
        format.html { redirect_to @url, notice: 'Url was successfully created.' }
        format.json { render action: 'show', status: :created, location: @url }
        wordcount
      else
        format.html { render action: 'new' }
        format.json { render json: @url.errors, status: :unprocessable_entity }
      end
    end
  end

  def wordcount
    # Choose the URL to visit
    @app_url = @url

    @words = HTTParty.get(@app_url)

    # Trick to pretty print headers
    @wordcount = Hash[*@words]
  end

  # PATCH/PUT /urls/1
  # PATCH/PUT /urls/1.json
  def update
    respond_to do |format|
      if @url.update(url_params)
        format.html { redirect_to @url, notice: 'Url was successfully updated.' }
        format.json { head :no_content }
      else
        format.html { render action: 'edit' }
        format.json { render json: @url.errors, status: :unprocessable_entity }
      end
    end
  end

  # DELETE /urls/1
  # DELETE /urls/1.json
  def destroy
    @url.destroy
    respond_to do |format|
      format.html { redirect_to urls_url }
      format.json { head :no_content }
    end
  end

  private
    # Use callbacks to share common setup or constraints between actions.
    def set_url
      @url = Url.find(params[:id])
    end

    # Never trust parameters from the scary internet, only allow the white list through.
    def url_params
      params.require(:url).permit(:url)
    end
end

That is my controller.rb. I am getting a 'bad argument (expected URI object or URI string)' from the line @words = HTTParty.get(@app_url) I need to change what url is put into the form to a valid URL, grab the content I want from that URL, and save that information.

Answer 1

Try something like this:

response = HTTParty.get('https://google.com')

puts response.body, response.code, response.message, response.headers.inspect

To answer your question you can implement the following method, make a new class or put it in a helper.

Might need to include HTTParty

def url_getter(url)
  HTTParty.get(url)
end

and call it:

url_getter(@app_url)