sklearn-pandas Archives

Python is a powerful general-purpose programming language.

Pandas: Knowing when an operation affects the original dataframe

January 15, 2021 Ollie MC

Question or problem about Python programming: I love pandas and have been using it for years and feel pretty confident I have a good handle on how to subset dataframes and deal with views vs copies appropriately (though I use a lot of assertions to be sure). I also know that there have been tons […]

Pandas in AWS lambda gives numpy error

January 15, 2021 Andrew Rocky

Question or problem about Python programming: I’ve been trying to run my code in AWS Lambda which imports pandas. So here is what I’ve done. I have a python file which contains a simple code as follows(This file has the lambda handler)

multiprocessing in python – sharing large object (e.g. pandas dataframe) between multiple processes

January 15, 2021 Abreonia Ng

Question or problem about Python programming: I am using Python multiprocessing, more precisely

pandas equivalent of np.where

January 15, 2021 Andrew Rocky

Question or problem about Python programming: np.where has the semantics of a vectorized if/else (similar to Apache Spark’s when/otherwise DataFrame method). I know that I can use np.where on pandas Series, but pandas often defines its own API to use instead of raw numpy functions, which is usually more convenient with pd.Series/pd.DataFrame.

How to convert OpenDocument spreadsheets to a pandas DataFrame?

January 15, 2021 Jeffrey Schneider

Question or problem about Python programming: The Python library pandas can read Excel spreadsheets and convert them to a pandas.DataFrame with pandas.read_excel(file) command. Under the hood, it uses xlrd library which does not support ods files.

Add new column in Pandas DataFrame Python

January 15, 2021 Abreonia Ng

Question or problem about Python programming: I have dataframe in Pandas for example: Col1 Col2 A 1 B 2 C 3 Now if I would like to add one more column named Col3 and the value is based on Col2. In formula, if Col2 > 1, then Col3 is 0, otherwise would be 1. So, […]

Putting many python pandas dataframes to one excel worksheet

January 15, 2021 Jeffrey Schneider

Question or problem about Python programming: It is quite easy to add many pandas dataframes into excel work book as long as it is different worksheets. But, it is somewhat tricky to get many dataframes into one worksheet if you want to use pandas built-in df.to_excel functionality.

How to group DataFrame by a period of time?

January 15, 2021 Odhran Miss

Question or problem about Python programming: I have some data from log files and would like to group entries by a minute:

Find unique values in a Pandas dataframe, irrespective of row or column location

January 14, 2021January 17, 2021 James Cameron

Question or problem about Python programming: I have a Pandas dataframe and I want to find all the unique values in that dataframe…irrespective of row/columns. If I have a 10 x 10 dataframe, and suppose they have 84 unique values, I need to find them – Not the count.

Check if dataframe column is Categorical

January 14, 2021January 17, 2021 Aba Tayler

Question or problem about Python programming: I can’t seem to get a simple dtype check working with Pandas’ improved Categoricals in v0.15+. Basically I just want something like is_categorical(column) -> True/False.