'DataFrame' object has no attribute 'sql' [on hold]2019 Community Moderator ElectionHow do I set/get heap size for Spark (via Python notebook)Spark DataFrame coursesRDD of gziped files to “uncompressed” DataframeDataframe request with groupBySpark SQL Pivot CrossTab functionalityPlot RDD data using a pyspark dataframe from csv filePySpark dataframe repartitionAdding multiple columns to spark dataframeSpark DataFrame Limit Function Takes Too Much Time to ShowHow to store dataframe result into text file?
Everything Bob says is false. How does he get people to trust him?
How to be diplomatic in refusing to write code that breaches the privacy of our users
Applicability of Single Responsibility Principle
Do I need a multiple entry visa for a trip UK -> Sweden -> UK?
Increase performance creating Mandelbrot set in python
Is there a problem with hiding "forgot password" until it's needed?
How to verify if g is a generator for p?
Opposite of a diet
Cynical novel that describes an America ruled by the media, arms manufacturers, and ethnic figureheads
There is only s̶i̶x̶t̶y one place he can be
How does it work when somebody invests in my business?
Bash method for viewing beginning and end of file
I'm in charge of equipment buying but no one's ever happy with what I choose. How to fix this?
Are there any comparative studies done between Ashtavakra Gita and Buddhim?
Best way to store options for panels
when is out of tune ok?
What defines a dissertation?
Why "be dealt cards" rather than "be dealing cards"?
Is there a good way to store credentials outside of a password manager?
How was Earth single-handedly capable of creating 3 of the 4 gods of chaos?
Finding all intervals that match predicate in vector
Implement the Thanos sorting algorithm
Star/Wye electrical connection math symbol
Was the picture area of a CRT a parallelogram (instead of a true rectangle)?
'DataFrame' object has no attribute 'sql' [on hold]
2019 Community Moderator ElectionHow do I set/get heap size for Spark (via Python notebook)Spark DataFrame coursesRDD of gziped files to “uncompressed” DataframeDataframe request with groupBySpark SQL Pivot CrossTab functionalityPlot RDD data using a pyspark dataframe from csv filePySpark dataframe repartitionAdding multiple columns to spark dataframeSpark DataFrame Limit Function Takes Too Much Time to ShowHow to store dataframe result into text file?
$begingroup$
I have compiled the following code in Databricks written in spark.sql. I'm getting an AttributeError: 'DataFrame' object has no attribute 'sql'
The code is as follows:
purchaseByCustomerPerHour = streamingDataFrame.sql("""SELECT
retail_data.CustomerID
,retail_data.InvoiceDate AS window
,SUM(retail_data.UnitPrice * retail_data.Quantity) AS total_cost
FROM retail_data
GROUP BY retail_data.CustomerID
,retail_data.InvoiceDate
,retail_data.UnitPrice""")
Normally, I would write the first line of code as follows:
purchaseByCustomerPerHour = spark.sql
However, I'm trying out StructuredStreaming with Spark.
Can someone interpret the error message?
Thanks
apache-spark pyspark
asked Mar 21 at 17:19
CarltonpCarltonp
1
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
put on hold as off-topic by Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F Mar 22 at 7:09
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question does not appear to be about data science, within the scope defined in the help center." – Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F
add a comment |
$begingroup$
I have compiled the following code in Databricks written in spark.sql. I'm getting an AttributeError: 'DataFrame' object has no attribute 'sql'
The code is as follows:
purchaseByCustomerPerHour = streamingDataFrame.sql("""SELECT
retail_data.CustomerID
,retail_data.InvoiceDate AS window
,SUM(retail_data.UnitPrice * retail_data.Quantity) AS total_cost
FROM retail_data
GROUP BY retail_data.CustomerID
,retail_data.InvoiceDate
,retail_data.UnitPrice""")
Normally, I would write the first line of code as follows:
purchaseByCustomerPerHour = spark.sql
However, I'm trying out StructuredStreaming with Spark.
Can someone interpret the error message?
Thanks
apache-spark pyspark
asked Mar 21 at 17:19
CarltonpCarltonp
1
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
put on hold as off-topic by Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F Mar 22 at 7:09
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question does not appear to be about data science, within the scope defined in the help center." – Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F
$begingroup$
I am running spark 2.4
$endgroup$
– Carltonp
Mar 21 at 17:21
$begingroup$
Your question is out of scope of this site. It's better to ask it in stackoverflow.com
$endgroup$
– Alireza Zolanvari
Mar 21 at 17:40
add a comment |
$begingroup$
I have compiled the following code in Databricks written in spark.sql. I'm getting an AttributeError: 'DataFrame' object has no attribute 'sql'
The code is as follows:
purchaseByCustomerPerHour = streamingDataFrame.sql("""SELECT
retail_data.CustomerID
,retail_data.InvoiceDate AS window
,SUM(retail_data.UnitPrice * retail_data.Quantity) AS total_cost
FROM retail_data
GROUP BY retail_data.CustomerID
,retail_data.InvoiceDate
,retail_data.UnitPrice""")
Normally, I would write the first line of code as follows:
purchaseByCustomerPerHour = spark.sql
However, I'm trying out StructuredStreaming with Spark.
Can someone interpret the error message?
Thanks
apache-spark pyspark
asked Mar 21 at 17:19
CarltonpCarltonp
1
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I have compiled the following code in Databricks written in spark.sql. I'm getting an AttributeError: 'DataFrame' object has no attribute 'sql'
The code is as follows:
purchaseByCustomerPerHour = streamingDataFrame.sql("""SELECT
retail_data.CustomerID
,retail_data.InvoiceDate AS window
,SUM(retail_data.UnitPrice * retail_data.Quantity) AS total_cost
FROM retail_data
GROUP BY retail_data.CustomerID
,retail_data.InvoiceDate
,retail_data.UnitPrice""")
Normally, I would write the first line of code as follows:
purchaseByCustomerPerHour = spark.sql
However, I'm trying out StructuredStreaming with Spark.
Can someone interpret the error message?
Thanks
apache-spark pyspark
apache-spark pyspark
asked Mar 21 at 17:19
CarltonpCarltonp
1
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Mar 21 at 17:19
CarltonpCarltonp
1
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Mar 21 at 17:19
CarltonpCarltonp
1
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked Mar 21 at 17:19
CarltonpCarltonp
1
asked Mar 21 at 17:19
CarltonpCarltonp
1
1
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Carltonp is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
put on hold as off-topic by Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F Mar 22 at 7:09
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question does not appear to be about data science, within the scope defined in the help center." – Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F
put on hold as off-topic by Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F Mar 22 at 7:09
This question appears to be off-topic. The users who voted to close gave this specific reason:
- "This question does not appear to be about data science, within the scope defined in the help center." – Kiritee Gak, Toros91, Siong Thye Goh, tuomastik, Mark.F
$begingroup$
I am running spark 2.4
$endgroup$
– Carltonp
Mar 21 at 17:21
$begingroup$
Your question is out of scope of this site. It's better to ask it in stackoverflow.com
$endgroup$
– Alireza Zolanvari
Mar 21 at 17:40
add a comment |
$begingroup$
I am running spark 2.4
$endgroup$
– Carltonp
Mar 21 at 17:21
$begingroup$
Your question is out of scope of this site. It's better to ask it in stackoverflow.com
$endgroup$
– Alireza Zolanvari
Mar 21 at 17:40
$begingroup$
I am running spark 2.4
$endgroup$
– Carltonp
Mar 21 at 17:21
$begingroup$
I am running spark 2.4
$endgroup$
– Carltonp
Mar 21 at 17:21
$begingroup$
Your question is out of scope of this site. It's better to ask it in stackoverflow.com
$endgroup$
– Alireza Zolanvari
Mar 21 at 17:40
$begingroup$
Your question is out of scope of this site. It's better to ask it in stackoverflow.com
$endgroup$
– Alireza Zolanvari
Mar 21 at 17:40
add a comment |
0
active
oldest
votes
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
I am running spark 2.4
$endgroup$
– Carltonp
Mar 21 at 17:21
$begingroup$
Your question is out of scope of this site. It's better to ask it in stackoverflow.com
$endgroup$
– Alireza Zolanvari
Mar 21 at 17:40